-- Logs begin at Thu 2022-07-21 08:28:15 UTC, end at Thu 2022-07-21 10:01:54 UTC. -- Jul 21 08:54:38.262996 dv-cinder-826608-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 21 08:54:38.272365 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 21 08:54:38.274448 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 21 08:54:38 2022] *** Jul 21 08:54:38.274448 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 21 08:54:38.274448 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 21 08:54:38.274448 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: nodename: dv-cinder-826608-6-iscsi Jul 21 08:54:38.274448 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: machine: x86_64 Jul 21 08:54:38.274448 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: clock source: unix Jul 21 08:54:38.274448 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: pcre jit disabled Jul 21 08:54:38.274448 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: detected number of CPU cores: 4 Jul 21 08:54:38.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: current working directory: / Jul 21 08:54:38.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: detected binary path: /usr/bin/uwsgi-core Jul 21 08:54:38.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: your processes number limit is 47680 Jul 21 08:54:38.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: your memory page size is 4096 bytes Jul 21 08:54:38.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: detected max file descriptor number: 2048 Jul 21 08:54:38.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: lock engine: pthread robust mutexes Jul 21 08:54:38.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: thunder lock: enabled Jul 21 08:54:38.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 21 08:54:38.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 21 08:54:38.323112 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: Python main interpreter initialized at 0x561770bea9c0 Jul 21 08:54:38.323112 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: python threads support enabled Jul 21 08:54:38.323112 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: your server socket listen backlog is limited to 100 connections Jul 21 08:54:38.323112 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: your mercy for graceful operations on workers is 90 seconds Jul 21 08:54:38.323501 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: mapped 403077 bytes (393 KB) for 2 cores Jul 21 08:54:38.323596 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: *** Operational MODE: preforking *** Jul 21 08:54:38.323908 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: *** uWSGI is running in multiple interpreter mode *** Jul 21 08:54:38.323908 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: spawned uWSGI master process (pid: 80186) Jul 21 08:54:38.324205 dv-cinder-826608-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 21 08:54:38.325854 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: spawned uWSGI worker 1 (pid: 80199, cores: 1) Jul 21 08:54:38.325854 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: spawned uWSGI worker 2 (pid: 80200, cores: 1) Jul 21 08:54:38.325854 dv-cinder-826608-6-iscsi devstack@keystone.service[80186]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 21 08:54:44.018378 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 08:54:44.018378 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: from cryptography.utils import int_from_bytes Jul 21 08:54:44.019540 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 08:54:44.019540 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: from cryptography.utils import int_from_bytes Jul 21 08:54:44.019748 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 08:54:44.019748 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: from cryptography.utils import int_from_bytes Jul 21 08:54:44.019748 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 08:54:44.019748 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: from cryptography.utils import int_from_bytes Jul 21 08:54:44.068784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 08:54:44.069083 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 08:54:44.069280 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] command line args: [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 08:54:44.069452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 08:54:44.069555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 08:54:44.069637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 08:54:44.069715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ================================================================================ {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 08:54:44.069821 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] command line args: [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 08:54:44.069954 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] admin_token = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.070091 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 08:54:44.070197 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] config_dir = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.070283 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ================================================================================ {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 08:54:44.070384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.070510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] admin_token = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.070602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] config_source = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.070733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] debug = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.070825 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] config_dir = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.070960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.071300 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.072355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] config_source = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.072355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] debug = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.072574 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.072574 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.072805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.072805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.073051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.073170 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.073289 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.073408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.073528 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] list_limit = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.073528 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] log_config_append = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.073758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.073871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.073984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] list_limit = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.074096 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] log_dir = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.074212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] log_config_append = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.074337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] log_file = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.074457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.074570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.074683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] log_dir = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.074683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] log_file = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.074908 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.075033 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.075182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.075380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.075563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.075758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.075990 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.075990 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.076353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.076353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.077399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.077584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.077766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.077966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.078156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.078355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.078544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.078737 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.078934 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.079121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.079300 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.079484 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.079665 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.079855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.080087 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.080273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.080455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.080638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.080638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.080973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.080973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] publish_errors = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.080973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.080973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.081648 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.081849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.081849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.082193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] publish_errors = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.082375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.082375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] standard_threads = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.082728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.082946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.083138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.083319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.083319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.083721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.083909 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] use_journal = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.083909 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] use_json = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.084269 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.084450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] use_stderr = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.084450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] use_syslog = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.084808 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.084987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.084987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.085328 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] standard_threads = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.085511 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.085511 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.085858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.086037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.086215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.086395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.086395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.086747 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.086926 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.087105 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] use_journal = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.087285 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.087463 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] use_json = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.087662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.087841 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] use_stderr = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.088033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.088033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.088383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] use_syslog = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.088566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.088753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:54:44.088942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.088942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.089287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.089473 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.089473 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.089834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.090013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.090013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.090347 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.090524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.090704 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.090891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.090891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.091225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.091411 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.091411 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.091753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.092196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.092391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.092577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.092577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.092896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.093056 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.093056 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.093414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.093595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.093595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.093941 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.094122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.094122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.094477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.094654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.094832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.095012 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.095012 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.095460 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.095641 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.095820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.096016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.096016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.096370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.096570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.096570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.096918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.097106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.097106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.097494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.097684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.097684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.098031 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.098208 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.098208 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.098539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.098737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.098915 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.099094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.099094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.099094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.099584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.099773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.099773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.100135 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.100315 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.100315 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.100779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.100971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.100971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.101587 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.101765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.101765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.102164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.102347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.102347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.102683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.102857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.103036 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.103230 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.103230 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.103569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.103770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.103770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.104266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.104471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.104471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.104812 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.104983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.104983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.105298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.105472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.105472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.105789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.105950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.106125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.106347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.106347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.106718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.106907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.106907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.107242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.107498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.107498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.107498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.107498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.108724 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.108894 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.108894 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.109216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.109396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.109396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.109396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.109852 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.110025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.110025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.110358 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.110552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.110552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.110887 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.111073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.111073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.111412 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.111605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.111605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.111961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.112134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.112134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.112472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.112676 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.112676 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.113007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.113187 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.113187 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.113539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.113719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.113719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.114050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.114231 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.114231 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.114592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.114784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.114784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] auth.external = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.115126 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.115305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.115305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.115653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.115835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.115835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] auth.password = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.116187 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.116348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] auth.token = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.116348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.116700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.116883 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.116883 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.117222 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.117411 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.117411 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.117764 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.117946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.117946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.118279 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.118457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] credential.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.118457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.118800 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.118982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.118982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.119318 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.119495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.119495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.120036 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.120239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.120239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.120593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.120773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.120773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.121112 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.121303 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.121303 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.121641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.121819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.121819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.122159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.122348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.122348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.122685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.122952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.122952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.123289 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.124100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.124100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.124422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.124586 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.124586 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.124891 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.125067 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.125067 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.125406 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.125595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.125595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.125934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.126125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.126125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.126463 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.126686 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.126686 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.127058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.127239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.127239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.127592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.127776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.127776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.128138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.128319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.128319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.128684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.128866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.128866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.129205 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.129388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.129388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.129757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.129940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.129940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.130280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.130475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.130475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.130843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.131027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.131027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.131364 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.131555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.131735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.131949 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.131949 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.131949 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.132451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.132634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.132634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.132982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.133161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.133161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.133501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.133681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.133681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.134038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.134221 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.134221 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.134567 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.134754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.134754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.135110 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.135302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.135302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.135635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.135816 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.135816 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.136191 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.136376 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.136376 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.136713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.136899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.136899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.137249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.137429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.137429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.137771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.137952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.137952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.138305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.138486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.138486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.138823 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.139488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.139488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.139823 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.140012 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.140012 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.140336 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.140588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.140588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.141057 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.141240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.141240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.141634 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.141823 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.142004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.142198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.142198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.142534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.142713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.142713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.143067 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.143258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.143258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.143595 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.143775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.143775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.144228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.144429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.144429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.144771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.144951 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.144951 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.145305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.145498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.145498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.145834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.146014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.146014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.146353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.146558 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.146558 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.146900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] auth.external = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.147079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.147079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.147420 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.147616 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.147616 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.147974 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.148142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.148142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.148467 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.148636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.148636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.148941 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] auth.password = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.149107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.149272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] auth.token = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.149449 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.149449 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.149766 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.149901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.149901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.150135 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.150274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.150274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.150513 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.150638 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.150638 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.150873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.151014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.151014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.151249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.151378 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.151378 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.151612 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.151753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.151881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.152031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.152031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.152264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] credential.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.152391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.152391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] role.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.152651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.152778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] role.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.152778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.153007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.153128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.153267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.153395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.153395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.153639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.153765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.153765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.154012 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.154140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.154268 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.154395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.154395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.154395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.155366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.155482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.155593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.155706 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.155706 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.155909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.156390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.156524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.156667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.156667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.156667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.157014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.157157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.157284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.157411 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.157411 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.157705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.157905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.157905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.158253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.158438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.158438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.158784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.158986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.158986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.159327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.159507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.159507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.159851 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.160074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.160074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.160416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.160598 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.160598 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.160939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.161132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.161132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.161476 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.161658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.161840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.162024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.162024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.162377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.162555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.162555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.162893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.163076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.163076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.163428 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.163607 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.163607 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.164005 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.164193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.164193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.164552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.164735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.164735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.165073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.165256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.165256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.165608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.166037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.166220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.166399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.166399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.connection = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.166762 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.166965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.166965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.167317 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.167499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.167499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.167835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.168053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.168053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.168397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.168576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.168576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.168915 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.169110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.169110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.169453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.169636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.169636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.169973 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.170336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.171257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.171445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.171445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.171787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.171989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.172200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.172381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.172381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.172720 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.172911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 08:54:44.173092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.173092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.173092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.173092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.173092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.173092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.173092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.173092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.173092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.174560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.174560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.174560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.174560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.174560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.174560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.174560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.174560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.174560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.176134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.176134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.176134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.176134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.176134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.176134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.176134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.176134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.176134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.177629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.177629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.177629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.177629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.177629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.177629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.177629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.177629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.177629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.179090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.179090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.179090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.179090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.179090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.179090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.179090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.179090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.179090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.180605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.180605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.180605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.180605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.180605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.180605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.180605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.180605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.180605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.182060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.182060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.182060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.182060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.182724 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.182724 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.183069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.183423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.183780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.184173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.184524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.184891 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.185234 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.185602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.185986 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.186340 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.186683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.187127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.187486 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.187836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.188234 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.188592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.188782 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.188968 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.189169 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.189354 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.189542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.189833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.190195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.190559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.190912 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.191271 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.191620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.192010 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.192360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] role.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.192717 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] role.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.193069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.193432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.193789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.194147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.194505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.194861 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.195257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.195605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.196000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.196344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.196705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.197061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.197426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.197709 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.197902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.198401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.198743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.199111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.199467 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.199827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.200233 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.200580 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.200944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.201316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.202318 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.203215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.203215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.203563 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.203870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.204274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.204634 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.204988 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.205339 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.205683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.206028 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.206386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.206747 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.207097 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.207490 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.207831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.208232 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.208600 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.208963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.209321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.209668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.210019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.210364 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.210732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.211090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.211435 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.211788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.212230 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.212599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.connection = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.212966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.213334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.213684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.214036 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.214386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.214732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.215079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.215431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.215790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.216176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.216533 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.216881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.217992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.218333 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.218689 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.219030 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.219384 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.219731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:54:44.220042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 08:54:44.227791 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.228063 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.247278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.247540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.247685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.256896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.276522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.276692 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.375222 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.411189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.428235 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.428532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.448041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.448295 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.496865 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.497062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.497272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.497386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.497520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.497602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.497727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.497882 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.497964 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.498071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.498170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.498775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.525473 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.525687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.525897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.526040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.526174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.526307 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.526469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.526654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.526798 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.526931 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.527069 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.527219 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.527219 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.527219 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.538666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.538895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.552644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.552892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.552892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.553147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.553147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.553399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.553563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.553701 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.553836 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.553971 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.554105 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.554255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.554391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.554513 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:54:44.572405 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.572600 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.572791 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 08:54:44.572910 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 08:54:44.573041 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.573125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.573194 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 08:54:44.573267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 08:54:44.573351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.573458 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.573592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.573722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.573850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.573976 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.574168 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.574305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.586056 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.586228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.586371 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.586478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.586562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.586690 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.586844 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.586942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.595027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.595531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.598345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.598617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.598894 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.599072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.599216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.599351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.599480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.599608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.599737 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.599880 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.600036 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.600170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.600303 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.600436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.600586 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.600715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.600851 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.600992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.601126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.601275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.601409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.601627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.628068 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.628510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.629726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.630008 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.630301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.630584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.635391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.635672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.635848 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.636048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.636190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.636419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.643266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.643545 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.643962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.644255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.648334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.648619 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.648895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.649075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.649219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.649419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.649566 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.649690 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.649806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.649958 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.650096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.650217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.650358 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.650507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.650754 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.651020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.669856 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.670249 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.670855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.671151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.675238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.675802 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.678890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.679186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.679575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.679873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.684196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.684881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.688390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.689136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.692166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.692468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.692753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.692890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.693049 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.693188 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.693322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.693458 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.693592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.693742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.693971 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.694246 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.712461 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.712852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.713121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.713317 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.713488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.713652 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.713818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.714060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.722041 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.722761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.723752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.724046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.724543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.724759 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.727077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.727351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.727626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.727784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.727942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.728094 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.728215 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.728371 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.728648 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.728961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.741724 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.742008 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.742288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.742572 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.742848 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.743125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.743226 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.743404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.743510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.743744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.744039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.744313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.744591 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.744878 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.763397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.764693 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.767060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.768006 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.770371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.771273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.772013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.772900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.773638 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.773915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.774192 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.774515 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.774793 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.775081 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.782537 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.783394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.784260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.785037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.787496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.787779 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.788079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.788341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.788492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.788632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.788894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.789172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.798706 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.798999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.799142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.799286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.799441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.799846 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.807152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.807430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.807711 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.808000 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.808112 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:54:44.808269 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.808356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.808541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.808625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.808805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.808885 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.809074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.809194 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.809341 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.809469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.809605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.809710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.809874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.810039 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.810197 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.810419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.810570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.811298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.811470 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.811711 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.811711 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.811711 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.812409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:54:44.849500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561770bea9c0 pid: 80199 (default app) Jul 21 08:54:44.850705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561770bea9c0 pid: 80200 (default app) Jul 21 08:54:44.853059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4e45b-0256-4aeb-b2c0-a6b7caf83f55 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:44.853354 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4e45b-0256-4aeb-b2c0-a6b7caf83f55 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:44.853637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4e45b-0256-4aeb-b2c0-a6b7caf83f55 None None] PATH_INFO: `/v3` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:44.858491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Thu Jul 21 08:54:44 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:54:53.037685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb98e8d-46da-423f-9fb7-19ea2531dc25 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:53.038255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb98e8d-46da-423f-9fb7-19ea2531dc25 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:53.038255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb98e8d-46da-423f-9fb7-19ea2531dc25 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:53.042865 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:54:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:54:53.049337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d98e5e-7836-4517-8e6f-184a2541e217 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:53.050061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d98e5e-7836-4517-8e6f-184a2541e217 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:53.050061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d98e5e-7836-4517-8e6f-184a2541e217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:53.122233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: INFO dbcounter [None req-f7d98e5e-7836-4517-8e6f-184a2541e217 None None] Registered counter for database keystone Jul 21 08:54:53.146366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG oslo_db.sqlalchemy.engines [None req-f7d98e5e-7836-4517-8e6f-184a2541e217 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80199) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 08:54:53.152940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writer thread running {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 08:54:53.345839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG passlib.handlers.bcrypt [None req-f7d98e5e-7836-4517-8e6f-184a2541e217 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80199) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 08:54:53.346168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG passlib.handlers.bcrypt [None req-f7d98e5e-7836-4517-8e6f-184a2541e217 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80199) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 08:54:53.401702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f7d98e5e-7836-4517-8e6f-184a2541e217 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:54:53.403387 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f7d98e5e-7836-4517-8e6f-184a2541e217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:54:53.535478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:54:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 21 08:54:53.707829 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb98e8d-46da-423f-9fb7-19ea2531dc25 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:54:53.782918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: INFO dbcounter [None req-ddb98e8d-46da-423f-9fb7-19ea2531dc25 None None] Registered counter for database keystone Jul 21 08:54:53.803559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG oslo_db.sqlalchemy.engines [None req-ddb98e8d-46da-423f-9fb7-19ea2531dc25 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80200) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 08:54:53.811159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writer thread running {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 08:54:53.936488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb98e8d-46da-423f-9fb7-19ea2531dc25 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:54:53.941021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b451cb7-5e93-4208-bada-b543b193e494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:54:53.941692 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b451cb7-5e93-4208-bada-b543b193e494 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:53.941977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b451cb7-5e93-4208-bada-b543b193e494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:53.942256 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b451cb7-5e93-4208-bada-b543b193e494 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:53.951823 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b451cb7-5e93-4208-bada-b543b193e494 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:54:54.008486 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9b451cb7-5e93-4208-bada-b543b193e494 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 08:54:54.008746 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b451cb7-5e93-4208-bada-b543b193e494 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:54:54.019753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Thu Jul 21 08:54:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 08:54:54.026654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d98e5e-7836-4517-8e6f-184a2541e217 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:54:54.053333 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d98e5e-7836-4517-8e6f-184a2541e217 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:54:54.061101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68ce89-e4f9-4622-a094-d2b87a5db0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:54:54.063270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68ce89-e4f9-4622-a094-d2b87a5db0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:54.063792 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68ce89-e4f9-4622-a094-d2b87a5db0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:54.064199 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68ce89-e4f9-4622-a094-d2b87a5db0d2 admin admin] PATH_INFO: `/v3/projects/a27dd5dc3697487bbb9b172396ff08c2` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:54.070447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68ce89-e4f9-4622-a094-d2b87a5db0d2 admin admin] RBAC: Authorizing `identity:get_project(project_id=a27dd5dc3697487bbb9b172396ff08c2)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:54:54.172908 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ad68ce89-e4f9-4622-a094-d2b87a5db0d2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 08:54:54.173534 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68ce89-e4f9-4622-a094-d2b87a5db0d2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:54:54.182893 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Thu Jul 21 08:54:54 2022] GET /identity/v3/projects/a27dd5dc3697487bbb9b172396ff08c2 => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:54:56.348960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f940d2a-5dfe-4849-8bf0-b1b3efaee8b2 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:56.349825 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f940d2a-5dfe-4849-8bf0-b1b3efaee8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:56.349982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f940d2a-5dfe-4849-8bf0-b1b3efaee8b2 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:56.354432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:54:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:54:56.361466 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020ddaf-4ab3-4dea-ab37-0e10f055752b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:56.362027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020ddaf-4ab3-4dea-ab37-0e10f055752b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:56.362027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020ddaf-4ab3-4dea-ab37-0e10f055752b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:56.385198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-9020ddaf-4ab3-4dea-ab37-0e10f055752b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:54:56.386875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9020ddaf-4ab3-4dea-ab37-0e10f055752b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:54:56.396687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:54:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 08:54:56.442866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f940d2a-5dfe-4849-8bf0-b1b3efaee8b2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:54:56.464336 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f940d2a-5dfe-4849-8bf0-b1b3efaee8b2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:54:56.473434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707a7bc-2eef-4304-b3dd-5e4095cd4523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:54:56.474569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707a7bc-2eef-4304-b3dd-5e4095cd4523 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:56.475086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707a7bc-2eef-4304-b3dd-5e4095cd4523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:56.475583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707a7bc-2eef-4304-b3dd-5e4095cd4523 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:56.491584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707a7bc-2eef-4304-b3dd-5e4095cd4523 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:54:56.495163 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1707a7bc-2eef-4304-b3dd-5e4095cd4523 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:54:56.514693 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Thu Jul 21 08:54:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:54:56.520902 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020ddaf-4ab3-4dea-ab37-0e10f055752b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:54:56.531222 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020ddaf-4ab3-4dea-ab37-0e10f055752b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:54:56.535729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a95842-dfe3-490d-8462-85e4769df6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:54:56.536650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a95842-dfe3-490d-8462-85e4769df6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:56.536937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a95842-dfe3-490d-8462-85e4769df6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:56.537218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a95842-dfe3-490d-8462-85e4769df6c1 admin admin] PATH_INFO: `/v3/users/af6fa97a676b4e53bc4e27a552d09f11` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:56.538679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a95842-dfe3-490d-8462-85e4769df6c1 admin admin] RBAC: Authorizing `identity:get_user(user_id=af6fa97a676b4e53bc4e27a552d09f11)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:54:56.540845 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a95842-dfe3-490d-8462-85e4769df6c1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:54:56.542501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Thu Jul 21 08:54:56 2022] GET /identity/v3/users/af6fa97a676b4e53bc4e27a552d09f11 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:54:58.899865 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805f2a7-7678-42a3-8a03-e1d63337c58e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:58.900731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805f2a7-7678-42a3-8a03-e1d63337c58e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:58.900731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805f2a7-7678-42a3-8a03-e1d63337c58e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:58.902103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:54:58.909909 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f668d-ee3e-4631-9e6e-3697aaa1ef87 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:58.910474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f668d-ee3e-4631-9e6e-3697aaa1ef87 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:58.910922 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f668d-ee3e-4631-9e6e-3697aaa1ef87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:58.938642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-bb7f668d-ee3e-4631-9e6e-3697aaa1ef87 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:54:58.939845 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bb7f668d-ee3e-4631-9e6e-3697aaa1ef87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:54:58.952006 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99d73d-7bb4-4ec8-804c-bc5b53d717b0 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:58.952304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99d73d-7bb4-4ec8-804c-bc5b53d717b0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:58.952586 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99d73d-7bb4-4ec8-804c-bc5b53d717b0 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:58.954044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:54:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:54:58.960054 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316c20d-26f6-4173-8007-68489129fcb8 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:58.960344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316c20d-26f6-4173-8007-68489129fcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:58.960636 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316c20d-26f6-4173-8007-68489129fcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:59.006470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG passlib.handlers.bcrypt [None req-9316c20d-26f6-4173-8007-68489129fcb8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80200) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 08:54:59.006793 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG passlib.handlers.bcrypt [None req-9316c20d-26f6-4173-8007-68489129fcb8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80200) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 08:54:59.039026 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:54:59.053361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-9316c20d-26f6-4173-8007-68489129fcb8 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:54:59.055004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9316c20d-26f6-4173-8007-68489129fcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:54:59.064420 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f668d-ee3e-4631-9e6e-3697aaa1ef87 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:54:59.093281 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:54:59.186211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f668d-ee3e-4631-9e6e-3697aaa1ef87 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:54:59.206624 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:54:59.223475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d464577-7bc0-4b87-a1a9-d742cfe6b9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:54:59.224317 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d464577-7bc0-4b87-a1a9-d742cfe6b9fe None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:59.224610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d464577-7bc0-4b87-a1a9-d742cfe6b9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:59.224896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d464577-7bc0-4b87-a1a9-d742cfe6b9fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:59.237907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d464577-7bc0-4b87-a1a9-d742cfe6b9fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:54:59.247026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316c20d-26f6-4173-8007-68489129fcb8 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-8d464577-7bc0-4b87-a1a9-d742cfe6b9fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:54:59.249927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:54:59.251658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:54:59.253167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:54:59.253167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:54:59.253167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:54:59.253167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:54:59.253167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:54:59.253167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:54:59.253167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:54:59.253167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:54:59.253167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:54:59.253167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:54:59.253167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:54:59.253167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:54:59.253167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:54:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:54:59.258851 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d464577-7bc0-4b87-a1a9-d742cfe6b9fe None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:54:59.360484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:54:59.384567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d464577-7bc0-4b87-a1a9-d742cfe6b9fe None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:54:59.387085 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316c20d-26f6-4173-8007-68489129fcb8 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:54:59.391318 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:54:59.438368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8307c1-2609-439a-8b91-b019fa36344a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:54:59.439122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8307c1-2609-439a-8b91-b019fa36344a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:59.439455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8307c1-2609-439a-8b91-b019fa36344a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:59.439831 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8307c1-2609-439a-8b91-b019fa36344a None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:59.440734 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8307c1-2609-439a-8b91-b019fa36344a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:54:59.441234 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8307c1-2609-439a-8b91-b019fa36344a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:54:59.442919 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8307c1-2609-439a-8b91-b019fa36344a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:54:59.449600 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecdff9-6c5f-4dea-a276-aa4f55c67277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:54:59.450696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecdff9-6c5f-4dea-a276-aa4f55c67277 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:59.451203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecdff9-6c5f-4dea-a276-aa4f55c67277 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:59.451723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecdff9-6c5f-4dea-a276-aa4f55c67277 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:59.457905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:54:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:54:59.464629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8307c1-2609-439a-8b91-b019fa36344a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:54:59.531731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecdff9-6c5f-4dea-a276-aa4f55c67277 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:54:59.533225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecdff9-6c5f-4dea-a276-aa4f55c67277 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:54:59.599494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8307c1-2609-439a-8b91-b019fa36344a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:54:59.649294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dd1f4-e4e3-427d-975d-44b97df2db95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:54:59.650312 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dd1f4-e4e3-427d-975d-44b97df2db95 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:59.650682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dd1f4-e4e3-427d-975d-44b97df2db95 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:59.651038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dd1f4-e4e3-427d-975d-44b97df2db95 None admin] PATH_INFO: `/v3/users/af6fa97a676b4e53bc4e27a552d09f11` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:59.652614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7dd1f4-e4e3-427d-975d-44b97df2db95 None admin] RBAC: Authorizing `identity:get_user(user_id=af6fa97a676b4e53bc4e27a552d09f11)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:54:59.654761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7dd1f4-e4e3-427d-975d-44b97df2db95 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:54:59.657238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 9/16] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:54:59 2022] GET /identity/v3/users/af6fa97a676b4e53bc4e27a552d09f11 => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:54:59.663217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dd1f4-e4e3-427d-975d-44b97df2db95 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:54:59.670591 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 8/17] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:54:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 08:54:59.677584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecdff9-6c5f-4dea-a276-aa4f55c67277 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:54:59.681470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:54:59.681663 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:54:59.807186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecdff9-6c5f-4dea-a276-aa4f55c67277 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:54:59.822244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dd1f4-e4e3-427d-975d-44b97df2db95 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:54:59.853857 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a2bb7-226f-4f39-8ff5-2533b2723d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:54:59.854771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a2bb7-226f-4f39-8ff5-2533b2723d65 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:59.855136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a2bb7-226f-4f39-8ff5-2533b2723d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:59.855511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a2bb7-226f-4f39-8ff5-2533b2723d65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:59.866381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a3eaa-d5e0-4a3c-83ab-b0af271c8006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:54:59.867266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a3eaa-d5e0-4a3c-83ab-b0af271c8006 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:54:59.867611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a3eaa-d5e0-4a3c-83ab-b0af271c8006 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:54:59.867992 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a3eaa-d5e0-4a3c-83ab-b0af271c8006 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:54:59.869311 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a3eaa-d5e0-4a3c-83ab-b0af271c8006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:54:59.870853 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a3eaa-d5e0-4a3c-83ab-b0af271c8006 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:54:59.873086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:54:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:54:59.879298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a3eaa-d5e0-4a3c-83ab-b0af271c8006 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:54:59.879577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a2bb7-226f-4f39-8ff5-2533b2723d65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:54:59.881238 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a2bb7-226f-4f39-8ff5-2533b2723d65 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:54:59.909072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-5d0a2bb7-226f-4f39-8ff5-2533b2723d65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:54:59.909072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:54:59.909072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:54:59.909072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:54:59.909072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:54:59.909072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:54:59.909072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:54:59.909072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:54:59.909072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:54:59.909072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:54:59.909072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:54:59.909072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:54:59.909072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:54:59.909072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:54:59.909072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:54:59.911355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:54:59.912932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:54:59.912932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:54:59.912932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:54:59.912932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:54:59.912932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:54:59.918025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a2bb7-226f-4f39-8ff5-2533b2723d65 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:54:59.989730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:54:59.999564 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:00.020724 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a3eaa-d5e0-4a3c-83ab-b0af271c8006 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:00.071157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a2bb7-226f-4f39-8ff5-2533b2723d65 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:00.073553 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbb8b9-573c-43f6-8da7-f3bb566b2dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:00.074582 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbb8b9-573c-43f6-8da7-f3bb566b2dfa None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:00.075034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbb8b9-573c-43f6-8da7-f3bb566b2dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:00.075360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbb8b9-573c-43f6-8da7-f3bb566b2dfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:00.077148 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbb8b9-573c-43f6-8da7-f3bb566b2dfa None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.domain.id=default, user.id=af6fa97a676b4e53bc4e27a552d09f11) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:00.077657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbb8b9-573c-43f6-8da7-f3bb566b2dfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:00.080209 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbb8b9-573c-43f6-8da7-f3bb566b2dfa None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:00.090684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1322 bytes} [Thu Jul 21 08:54:59 2022] GET /identity/v3/role_assignments?user.id=af6fa97a676b4e53bc4e27a552d09f11&scope.domain.id=default&role.id=000e942517f2469ba36d2e65978bc314 => generated 233 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:00.120723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec70779-2768-48e7-bad1-bef3177534a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:00.121527 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec70779-2768-48e7-bad1-bef3177534a2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:00.121896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec70779-2768-48e7-bad1-bef3177534a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:00.122274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec70779-2768-48e7-bad1-bef3177534a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:00.123198 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec70779-2768-48e7-bad1-bef3177534a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:00.123595 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec70779-2768-48e7-bad1-bef3177534a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:00.125248 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec70779-2768-48e7-bad1-bef3177534a2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:00.148014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:03.557920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b99d7-7a06-447a-9375-65f996140d2d None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:03.558474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b99d7-7a06-447a-9375-65f996140d2d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:03.558907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b99d7-7a06-447a-9375-65f996140d2d None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:03.562535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:03.571205 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb965bf4-cfe0-4b62-a7da-db2b47eabfdf None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:03.572695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb965bf4-cfe0-4b62-a7da-db2b47eabfdf None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:03.573497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb965bf4-cfe0-4b62-a7da-db2b47eabfdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:03.616131 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-eb965bf4-cfe0-4b62-a7da-db2b47eabfdf None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:03.618649 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-eb965bf4-cfe0-4b62-a7da-db2b47eabfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:03.678432 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee2d32-2719-4655-a945-a2570f37b384 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:03.679366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee2d32-2719-4655-a945-a2570f37b384 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:03.684303 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee2d32-2719-4655-a945-a2570f37b384 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:03.688674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 13/23] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:03 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:03.695937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0e541-1989-4aa0-beac-4cf5b9f55e42 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:03.696888 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0e541-1989-4aa0-beac-4cf5b9f55e42 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:03.697654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0e541-1989-4aa0-beac-4cf5b9f55e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:03.735309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 11/24] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:03.765141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-34d0e541-1989-4aa0-beac-4cf5b9f55e42 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:03.766371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-34d0e541-1989-4aa0-beac-4cf5b9f55e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:03.806268 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb965bf4-cfe0-4b62-a7da-db2b47eabfdf None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:03.898717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 14/25] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:03.929878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0e541-1989-4aa0-beac-4cf5b9f55e42 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:03.952308 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:04.098043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:04.124183 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb965bf4-cfe0-4b62-a7da-db2b47eabfdf None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:04.157204 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0e541-1989-4aa0-beac-4cf5b9f55e42 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:04.209002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45c40a-f9ad-4f0c-b9f6-1844f0a52a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:04.209675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45c40a-f9ad-4f0c-b9f6-1844f0a52a1b None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:04.209963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45c40a-f9ad-4f0c-b9f6-1844f0a52a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:04.210246 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45c40a-f9ad-4f0c-b9f6-1844f0a52a1b None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:04.211034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45c40a-f9ad-4f0c-b9f6-1844f0a52a1b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:04.212518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45c40a-f9ad-4f0c-b9f6-1844f0a52a1b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:04.223379 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6140d-8c4f-45aa-8b94-f8c2028c2ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:04.224225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6140d-8c4f-45aa-8b94-f8c2028c2ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:04.224520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6140d-8c4f-45aa-8b94-f8c2028c2ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:04.224802 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6140d-8c4f-45aa-8b94-f8c2028c2ed3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:04.248682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6140d-8c4f-45aa-8b94-f8c2028c2ed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-3fa6140d-8c4f-45aa-8b94-f8c2028c2ed3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:04.258203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:04.259715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:04.261178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:04.261178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:04.261178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:04.261178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:04.261178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:04.261178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:04.261178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:04.261178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:04.261178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:04.261178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:04.261178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:04.261178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:04.261178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 12/26] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:55:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 454 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:04.261178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca19302-54bc-4f79-9c7d-f7101bb7a4ed None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:04.262443 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca19302-54bc-4f79-9c7d-f7101bb7a4ed None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:04.262443 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca19302-54bc-4f79-9c7d-f7101bb7a4ed None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:04.264165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 13/27] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:04.266706 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4edfc-a51a-4c80-8aa8-5eee4f0956e1 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:04.266706 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4edfc-a51a-4c80-8aa8-5eee4f0956e1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:04.266706 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4edfc-a51a-4c80-8aa8-5eee4f0956e1 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:04.266706 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 14/28] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:04.273019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc66d-bcca-446a-a91b-de0051cffdf2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:04.273572 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc66d-bcca-446a-a91b-de0051cffdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:04.273993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc66d-bcca-446a-a91b-de0051cffdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:04.293406 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c5fdc66d-bcca-446a-a91b-de0051cffdf2 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:04.295411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c5fdc66d-bcca-446a-a91b-de0051cffdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:04.383948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 15/29] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Jul 21 08:55:03 2022] POST /identity/v3/roles => generated 219 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:55:04.383948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45c40a-f9ad-4f0c-b9f6-1844f0a52a1b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:04.389795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 15/30] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:04.391535 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868a1fe-1055-4e95-a4f1-1732e48c9fda None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:04.392072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868a1fe-1055-4e95-a4f1-1732e48c9fda None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:04.392592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868a1fe-1055-4e95-a4f1-1732e48c9fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:04.429441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-8868a1fe-1055-4e95-a4f1-1732e48c9fda None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:04.430952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8868a1fe-1055-4e95-a4f1-1732e48c9fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:04.468934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:04.515609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 16/31] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:04.516956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868a1fe-1055-4e95-a4f1-1732e48c9fda None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:04.556337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45c40a-f9ad-4f0c-b9f6-1844f0a52a1b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:04.606148 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f06458-6159-45fe-96a0-4a8367fe31f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:04.607620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f06458-6159-45fe-96a0-4a8367fe31f5 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:04.607968 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f06458-6159-45fe-96a0-4a8367fe31f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:04.608441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f06458-6159-45fe-96a0-4a8367fe31f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:04.609255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f06458-6159-45fe-96a0-4a8367fe31f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:04.609575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f06458-6159-45fe-96a0-4a8367fe31f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:04.611295 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f06458-6159-45fe-96a0-4a8367fe31f5 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:04.618095 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:04.623019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 16/32] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:04.638757 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f06458-6159-45fe-96a0-4a8367fe31f5 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:04.645687 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868a1fe-1055-4e95-a4f1-1732e48c9fda None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:04.714735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116928b-c38f-4201-bd6e-35357b8787c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:04.716976 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116928b-c38f-4201-bd6e-35357b8787c4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:04.717505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116928b-c38f-4201-bd6e-35357b8787c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:04.718042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116928b-c38f-4201-bd6e-35357b8787c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:04.739627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116928b-c38f-4201-bd6e-35357b8787c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:04.741560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116928b-c38f-4201-bd6e-35357b8787c4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:04.772772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-7116928b-c38f-4201-bd6e-35357b8787c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:55:04.772772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:04.772772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:04.772772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:04.772772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:04.772772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:04.772772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:04.772772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:04.772772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:04.772772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:04.772772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:04.772772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:04.772772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:55:04.772772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:55:04.772772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:04.773637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:04.774424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:55:04.774424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:55:04.774424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:55:04.774424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:04.780899 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 17/33] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:04.782845 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116928b-c38f-4201-bd6e-35357b8787c4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:04.785206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f06458-6159-45fe-96a0-4a8367fe31f5 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:04.823300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:04.867737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b05d48-b0f3-4007-b5c0-e652f6282387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:04.868989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b05d48-b0f3-4007-b5c0-e652f6282387 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:04.869512 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b05d48-b0f3-4007-b5c0-e652f6282387 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:04.870025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b05d48-b0f3-4007-b5c0-e652f6282387 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:04.871484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b05d48-b0f3-4007-b5c0-e652f6282387 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:04.875057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b05d48-b0f3-4007-b5c0-e652f6282387 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:04.907530 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 17/34] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Jul 21 08:55:04 2022] POST /identity/v3/roles => generated 225 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:55:04.909916 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b05d48-b0f3-4007-b5c0-e652f6282387 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:04.937175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116928b-c38f-4201-bd6e-35357b8787c4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:04.982305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ab212-a78f-4e71-8a22-f72b92e82ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:04.983799 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ab212-a78f-4e71-8a22-f72b92e82ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:04.984375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ab212-a78f-4e71-8a22-f72b92e82ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:04.984847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ab212-a78f-4e71-8a22-f72b92e82ae9 None admin] PATH_INFO: `/v3/users/af6fa97a676b4e53bc4e27a552d09f11` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:04.986851 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ab212-a78f-4e71-8a22-f72b92e82ae9 None admin] RBAC: Authorizing `identity:get_user(user_id=af6fa97a676b4e53bc4e27a552d09f11)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:04.988688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ab212-a78f-4e71-8a22-f72b92e82ae9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:04.990994 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 18/35] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:04 2022] GET /identity/v3/users/af6fa97a676b4e53bc4e27a552d09f11 => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:04.996892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ab212-a78f-4e71-8a22-f72b92e82ae9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:05.050973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:05.148097 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b05d48-b0f3-4007-b5c0-e652f6282387 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:05.218343 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf236dc5-8848-4b81-acee-081cd1add70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:05.219520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf236dc5-8848-4b81-acee-081cd1add70f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:05.220062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf236dc5-8848-4b81-acee-081cd1add70f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:05.220580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf236dc5-8848-4b81-acee-081cd1add70f None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:05.222026 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf236dc5-8848-4b81-acee-081cd1add70f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:05.222576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf236dc5-8848-4b81-acee-081cd1add70f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:05.225427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf236dc5-8848-4b81-acee-081cd1add70f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:05.261226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 18/36] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:05.265464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:05.271626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf236dc5-8848-4b81-acee-081cd1add70f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:05.282750 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ab212-a78f-4e71-8a22-f72b92e82ae9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:05.337234 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca379fe-741f-4c3a-a770-4f81a0f98eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:05.338048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca379fe-741f-4c3a-a770-4f81a0f98eff None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:05.338453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca379fe-741f-4c3a-a770-4f81a0f98eff None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:05.338741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca379fe-741f-4c3a-a770-4f81a0f98eff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:05.340163 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca379fe-741f-4c3a-a770-4f81a0f98eff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:05.342296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca379fe-741f-4c3a-a770-4f81a0f98eff None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:05.344523 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 19/37] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:55:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:05.351594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca379fe-741f-4c3a-a770-4f81a0f98eff None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:05.482375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca379fe-741f-4c3a-a770-4f81a0f98eff None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:05.493780 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf236dc5-8848-4b81-acee-081cd1add70f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:05.519629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:05.539255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ded67-dee0-41f4-a540-206f879b2b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:05.540448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ded67-dee0-41f4-a540-206f879b2b19 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:05.540806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ded67-dee0-41f4-a540-206f879b2b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:05.541176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ded67-dee0-41f4-a540-206f879b2b19 None admin] PATH_INFO: `/v3/domains/default/users/af6fa97a676b4e53bc4e27a552d09f11/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:05.543322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ded67-dee0-41f4-a540-206f879b2b19 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=af6fa97a676b4e53bc4e27a552d09f11, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:05.544883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ded67-dee0-41f4-a540-206f879b2b19 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:05.545363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63edbd-e837-4f18-a6ac-ac5ffd65c2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:05.546046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63edbd-e837-4f18-a6ac-ac5ffd65c2b6 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:05.546335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63edbd-e837-4f18-a6ac-ac5ffd65c2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:05.546618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63edbd-e837-4f18-a6ac-ac5ffd65c2b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:05.547611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63edbd-e837-4f18-a6ac-ac5ffd65c2b6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:05.549159 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63edbd-e837-4f18-a6ac-ac5ffd65c2b6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:05.588434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 19/38] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jul 21 08:55:05 2022] POST /identity/v3/projects => generated 301 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:55:05.591581 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e62af-344e-4fc1-9ac8-2a371dd7892e None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:05.592458 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e62af-344e-4fc1-9ac8-2a371dd7892e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:05.593237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e62af-344e-4fc1-9ac8-2a371dd7892e None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:05.596173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 20/39] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:05.606108 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a75e6-5573-4858-bf2a-3d529ff13473 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:05.606563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a75e6-5573-4858-bf2a-3d529ff13473 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:05.607166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a75e6-5573-4858-bf2a-3d529ff13473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:05.630889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5e0a75e6-5573-4858-bf2a-3d529ff13473 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:05.633014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5e0a75e6-5573-4858-bf2a-3d529ff13473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:05.725126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 21/40] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:05.727767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7fb2a-0f2c-4de8-8cac-c23cc414ee43 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:05.728550 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7fb2a-0f2c-4de8-8cac-c23cc414ee43 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:05.729194 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7fb2a-0f2c-4de8-8cac-c23cc414ee43 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:05.732485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 22/41] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:05.742500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d2d99-2022-4c3f-91f2-43a98d572263 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:05.742876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d2d99-2022-4c3f-91f2-43a98d572263 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:05.743392 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d2d99-2022-4c3f-91f2-43a98d572263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:05.759710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 20/42] 10.222.0.13 () {58 vars in 1361 bytes} [Thu Jul 21 08:55:05 2022] PUT /identity/v3/domains/default/users/af6fa97a676b4e53bc4e27a552d09f11/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:55:05.763552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ded67-dee0-41f4-a540-206f879b2b19 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:05.798437 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:05.938631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-596d2d99-2022-4c3f-91f2-43a98d572263 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:05.941008 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-596d2d99-2022-4c3f-91f2-43a98d572263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:05.992521 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ded67-dee0-41f4-a540-206f879b2b19 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:06.048674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 23/43] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:06.058806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a3c53-ff88-4e06-9b1c-442a52778469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:06.059505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a3c53-ff88-4e06-9b1c-442a52778469 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:06.060360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a3c53-ff88-4e06-9b1c-442a52778469 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:06.060642 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a3c53-ff88-4e06-9b1c-442a52778469 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:06.061419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a3c53-ff88-4e06-9b1c-442a52778469 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:06.063044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a3c53-ff88-4e06-9b1c-442a52778469 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:06.085512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 21/44] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Jul 21 08:55:05 2022] POST /identity/v3/roles => generated 223 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:55:06.095531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d2d99-2022-4c3f-91f2-43a98d572263 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:06.156415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:06.305847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d2d99-2022-4c3f-91f2-43a98d572263 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:06.367181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026e91e-e0a6-43d1-bad0-303f0e20df1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:06.368567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026e91e-e0a6-43d1-bad0-303f0e20df1a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:06.369078 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026e91e-e0a6-43d1-bad0-303f0e20df1a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:06.369584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026e91e-e0a6-43d1-bad0-303f0e20df1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:06.371694 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026e91e-e0a6-43d1-bad0-303f0e20df1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:06.375100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026e91e-e0a6-43d1-bad0-303f0e20df1a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:06.378257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 24/45] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:55:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:06.384361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a3c53-ff88-4e06-9b1c-442a52778469 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:06.493056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:06.525404 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a3c53-ff88-4e06-9b1c-442a52778469 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:06.566276 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cc1de-0e94-4828-94ca-ff0cac166b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:06.566849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cc1de-0e94-4828-94ca-ff0cac166b4d None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:06.567137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cc1de-0e94-4828-94ca-ff0cac166b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:06.567423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cc1de-0e94-4828-94ca-ff0cac166b4d None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:06.568283 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cc1de-0e94-4828-94ca-ff0cac166b4d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:06.569784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cc1de-0e94-4828-94ca-ff0cac166b4d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:06.603543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 22/46] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jul 21 08:55:06 2022] POST /identity/v3/projects => generated 312 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:55:07.945074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e1a91-d50f-458a-9325-429fece25c4f None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:07.945611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e1a91-d50f-458a-9325-429fece25c4f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:07.945917 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134e1a91-d50f-458a-9325-429fece25c4f None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:07.948272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 25/47] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:07.958809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d2659-e06b-4c9f-b926-38a303f9457e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:07.959431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d2659-e06b-4c9f-b926-38a303f9457e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:07.960076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d2659-e06b-4c9f-b926-38a303f9457e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:07.988290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-8a7d2659-e06b-4c9f-b926-38a303f9457e None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:07.990661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8a7d2659-e06b-4c9f-b926-38a303f9457e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:08.087409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 23/48] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:55:08.111582 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e1a91-d50f-458a-9325-429fece25c4f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:08.216553 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:08.224436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134e1a91-d50f-458a-9325-429fece25c4f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:08.261754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d778fef-d168-4869-9f52-18508892540d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:08.262554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d778fef-d168-4869-9f52-18508892540d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:08.262952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d778fef-d168-4869-9f52-18508892540d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:08.263594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d778fef-d168-4869-9f52-18508892540d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:08.275488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d778fef-d168-4869-9f52-18508892540d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-7d778fef-d168-4869-9f52-18508892540d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:08.283661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:08.284611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:08.285340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:08.285340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:08.285340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:08.285340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:08.285340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:08.285340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:08.285340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:08.285340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:08.285340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:08.285340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:08.285340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:08.285340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:08.285340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 26/49] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:55:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:08.291513 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d2659-e06b-4c9f-b926-38a303f9457e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:08.355558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:08.409753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d2659-e06b-4c9f-b926-38a303f9457e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:08.448870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43d037-34d7-4805-8945-05a42baf9a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:08.449558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43d037-34d7-4805-8945-05a42baf9a27 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:08.449845 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43d037-34d7-4805-8945-05a42baf9a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:08.450658 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43d037-34d7-4805-8945-05a42baf9a27 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:08.451428 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43d037-34d7-4805-8945-05a42baf9a27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:08.451856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43d037-34d7-4805-8945-05a42baf9a27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:08.453408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43d037-34d7-4805-8945-05a42baf9a27 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:08.467643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 24/50] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:08.473577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d778fef-d168-4869-9f52-18508892540d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:08.610700 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d778fef-d168-4869-9f52-18508892540d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:08.664189 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399e027-c769-42b8-84c2-0fdfea9cfb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:08.665495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399e027-c769-42b8-84c2-0fdfea9cfb2b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:08.665869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399e027-c769-42b8-84c2-0fdfea9cfb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:08.666238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399e027-c769-42b8-84c2-0fdfea9cfb2b None admin] PATH_INFO: `/v3/users/af6fa97a676b4e53bc4e27a552d09f11` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:08.667776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399e027-c769-42b8-84c2-0fdfea9cfb2b None admin] RBAC: Authorizing `identity:get_user(user_id=af6fa97a676b4e53bc4e27a552d09f11)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:08.669448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399e027-c769-42b8-84c2-0fdfea9cfb2b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:08.671343 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 27/51] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:08 2022] GET /identity/v3/users/af6fa97a676b4e53bc4e27a552d09f11 => generated 247 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:08.677190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43d037-34d7-4805-8945-05a42baf9a27 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:08.811702 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43d037-34d7-4805-8945-05a42baf9a27 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:08.885298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73b4f3-92c8-4783-b878-e051257b5783 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:08.885723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73b4f3-92c8-4783-b878-e051257b5783 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:08.886224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73b4f3-92c8-4783-b878-e051257b5783 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:08.889414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 28/52] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:08.896648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd07dde-bc93-49d4-becc-be87b27b749c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:08.897135 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd07dde-bc93-49d4-becc-be87b27b749c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:08.897626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd07dde-bc93-49d4-becc-be87b27b749c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:08.926935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3fd07dde-bc93-49d4-becc-be87b27b749c None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:08.928374 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3fd07dde-bc93-49d4-becc-be87b27b749c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:08.941705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:08.954017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:08.974354 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622e212-c1ec-4a57-80cf-dd7dc158e4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:08.975863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622e212-c1ec-4a57-80cf-dd7dc158e4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:08.976410 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622e212-c1ec-4a57-80cf-dd7dc158e4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:08.976916 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622e212-c1ec-4a57-80cf-dd7dc158e4d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:08.979070 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622e212-c1ec-4a57-80cf-dd7dc158e4d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:08.982163 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622e212-c1ec-4a57-80cf-dd7dc158e4d0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:08.992686 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 25/53] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:55:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:08.993898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622e212-c1ec-4a57-80cf-dd7dc158e4d0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:09.015033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 29/54] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:09.053781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd07dde-bc93-49d4-becc-be87b27b749c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:09.113296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622e212-c1ec-4a57-80cf-dd7dc158e4d0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:09.151521 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d86207-e8b7-4d76-80b3-cb9c3ad3caba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:09.152228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d86207-e8b7-4d76-80b3-cb9c3ad3caba None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:09.152516 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d86207-e8b7-4d76-80b3-cb9c3ad3caba None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:09.152795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d86207-e8b7-4d76-80b3-cb9c3ad3caba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:09.154363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d86207-e8b7-4d76-80b3-cb9c3ad3caba None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.domain.id=default, user.id=af6fa97a676b4e53bc4e27a552d09f11) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:09.154755 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d86207-e8b7-4d76-80b3-cb9c3ad3caba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:09.156439 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d86207-e8b7-4d76-80b3-cb9c3ad3caba None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:09.165966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 26/55] 10.222.0.13 () {56 vars in 1322 bytes} [Thu Jul 21 08:55:08 2022] GET /identity/v3/role_assignments?user.id=af6fa97a676b4e53bc4e27a552d09f11&scope.domain.id=default&role.id=000e942517f2469ba36d2e65978bc314 => generated 530 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:09.187035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd07dde-bc93-49d4-becc-be87b27b749c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:09.232562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c2cc9-fc9c-46bc-bfdc-855ee35fdbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:09.233512 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c2cc9-fc9c-46bc-bfdc-855ee35fdbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:09.233880 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c2cc9-fc9c-46bc-bfdc-855ee35fdbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:09.234240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c2cc9-fc9c-46bc-bfdc-855ee35fdbd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:09.235718 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c2cc9-fc9c-46bc-bfdc-855ee35fdbd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:09.237448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c2cc9-fc9c-46bc-bfdc-855ee35fdbd0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:09.239384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 30/56] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:55:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:09.249162 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d86207-e8b7-4d76-80b3-cb9c3ad3caba None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:09.314805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:09.395855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d86207-e8b7-4d76-80b3-cb9c3ad3caba None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:09.449504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d3a30-fac8-4797-ac9b-720b3bf5cb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:09.450515 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d3a30-fac8-4797-ac9b-720b3bf5cb00 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:09.450869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d3a30-fac8-4797-ac9b-720b3bf5cb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:09.451235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d3a30-fac8-4797-ac9b-720b3bf5cb00 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:09.452028 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d3a30-fac8-4797-ac9b-720b3bf5cb00 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:09.453558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d3a30-fac8-4797-ac9b-720b3bf5cb00 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:09.484805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 27/57] 10.222.0.13 () {60 vars in 1140 bytes} [Thu Jul 21 08:55:09 2022] POST /identity/v3/projects => generated 298 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:55:11.614781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a045f-fbd0-4c70-a90a-cdd5e5fb749d None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:11.615311 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a045f-fbd0-4c70-a90a-cdd5e5fb749d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:11.615416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a045f-fbd0-4c70-a90a-cdd5e5fb749d None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:11.616770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 31/58] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:55:11.623064 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65511877-dc83-40c4-8d42-e527e4f30adf None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:11.623525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65511877-dc83-40c4-8d42-e527e4f30adf None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:11.623696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65511877-dc83-40c4-8d42-e527e4f30adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:11.642001 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-65511877-dc83-40c4-8d42-e527e4f30adf None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:11.643249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-65511877-dc83-40c4-8d42-e527e4f30adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:11.715497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 28/59] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:11.739751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a045f-fbd0-4c70-a90a-cdd5e5fb749d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:11.767580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:11.856758 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a045f-fbd0-4c70-a90a-cdd5e5fb749d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:11.897747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9aef75-767e-41b2-8e27-d975ef308d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:11.898554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aef75-767e-41b2-8e27-d975ef308d9f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:11.898844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aef75-767e-41b2-8e27-d975ef308d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:11.899128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aef75-767e-41b2-8e27-d975ef308d9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:11.900377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9aef75-767e-41b2-8e27-d975ef308d9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:11.901859 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9aef75-767e-41b2-8e27-d975ef308d9f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:11.904190 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 32/60] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:55:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:11.910160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65511877-dc83-40c4-8d42-e527e4f30adf None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:11.943344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:12.022401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65511877-dc83-40c4-8d42-e527e4f30adf None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:12.068260 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3152a-d4cb-4efe-bd32-3fe863911376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:12.069396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3152a-d4cb-4efe-bd32-3fe863911376 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:12.069921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3152a-d4cb-4efe-bd32-3fe863911376 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:12.070419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3152a-d4cb-4efe-bd32-3fe863911376 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:12.071765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3152a-d4cb-4efe-bd32-3fe863911376 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:12.074533 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3152a-d4cb-4efe-bd32-3fe863911376 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:12.112634 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 29/61] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:55:11 2022] POST /identity/v3/users => generated 275 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:55:16.041720 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c89d5-23af-4ca6-b1a0-eea4764acca3 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:16.042643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c89d5-23af-4ca6-b1a0-eea4764acca3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:16.043096 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c89d5-23af-4ca6-b1a0-eea4764acca3 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:16.044890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 33/62] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:16.050856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc31d44-edbc-4840-97b9-f0160b511408 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:16.051734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc31d44-edbc-4840-97b9-f0160b511408 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:16.052476 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc31d44-edbc-4840-97b9-f0160b511408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:16.078563 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3dc31d44-edbc-4840-97b9-f0160b511408 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:16.080100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3dc31d44-edbc-4840-97b9-f0160b511408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:16.171334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9e290-39cf-4d75-bafd-f4ba3f934e42 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:16.171751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9e290-39cf-4d75-bafd-f4ba3f934e42 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:16.171995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9e290-39cf-4d75-bafd-f4ba3f934e42 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:16.173291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 34/63] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:55:16.182396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ee6fc-7ea2-42ba-83de-6779570f04f8 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:16.183351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ee6fc-7ea2-42ba-83de-6779570f04f8 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:16.184668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ee6fc-7ea2-42ba-83de-6779570f04f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:16.194818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 30/64] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:16.211258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-bb6ee6fc-7ea2-42ba-83de-6779570f04f8 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:16.212497 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bb6ee6fc-7ea2-42ba-83de-6779570f04f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:16.219836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc31d44-edbc-4840-97b9-f0160b511408 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:16.253637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:16.335195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 35/65] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:16.413045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ee6fc-7ea2-42ba-83de-6779570f04f8 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:16.460849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:16.464354 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc31d44-edbc-4840-97b9-f0160b511408 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:16.510173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1ae8d-1637-4057-b794-c5fafb573c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:16.510992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1ae8d-1637-4057-b794-c5fafb573c11 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:16.511281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1ae8d-1637-4057-b794-c5fafb573c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:16.511570 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1ae8d-1637-4057-b794-c5fafb573c11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:16.535370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1ae8d-1637-4057-b794-c5fafb573c11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-b1e1ae8d-1637-4057-b794-c5fafb573c11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:16.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:16.550605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:16.553084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:16.553084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:16.553084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:16.553084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:16.553084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:16.553084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:16.553084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:16.553084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:16.553084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:16.553084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:16.553084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:16.553084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:16.553084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 31/66] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:55:16.554296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc49307-6396-40a4-a0c8-a6d9488a9b52 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:16.554296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc49307-6396-40a4-a0c8-a6d9488a9b52 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:16.554524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc49307-6396-40a4-a0c8-a6d9488a9b52 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:16.560729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 32/67] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:16.561703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc49307-6396-40a4-a0c8-a6d9488a9b52 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:16.605654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ee6fc-7ea2-42ba-83de-6779570f04f8 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:16.646463 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f23d2-51ac-415a-8969-be2e440fc348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:16.647551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f23d2-51ac-415a-8969-be2e440fc348 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:16.648049 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f23d2-51ac-415a-8969-be2e440fc348 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:16.648475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f23d2-51ac-415a-8969-be2e440fc348 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:16.661215 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f23d2-51ac-415a-8969-be2e440fc348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-495f23d2-51ac-415a-8969-be2e440fc348 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:16.670982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:16.671900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:16.672693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:16.672693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:16.672693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:16.672693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:16.672693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:16.672693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:16.672693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:16.672693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:16.672693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:16.672693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:16.672693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:55:16.672693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:16.675464 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 36/68] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:55:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:55:16.677172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8ec22-f50d-4547-9226-084d87e64945 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:16.677671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8ec22-f50d-4547-9226-084d87e64945 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:16.678144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8ec22-f50d-4547-9226-084d87e64945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:16.697709 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d6d8ec22-f50d-4547-9226-084d87e64945 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:16.699237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d6d8ec22-f50d-4547-9226-084d87e64945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:16.741472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:16.778998 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 37/69] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:16.779713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8ec22-f50d-4547-9226-084d87e64945 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:16.784356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc49307-6396-40a4-a0c8-a6d9488a9b52 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:16.796868 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:16.853492 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d655c-0429-4955-98a3-f4fd56e9680a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:16.855176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d655c-0429-4955-98a3-f4fd56e9680a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:16.856275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d655c-0429-4955-98a3-f4fd56e9680a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:16.857199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d655c-0429-4955-98a3-f4fd56e9680a None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:16.858888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d655c-0429-4955-98a3-f4fd56e9680a None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:16.859870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d655c-0429-4955-98a3-f4fd56e9680a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:16.862946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d655c-0429-4955-98a3-f4fd56e9680a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:16.888984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 33/70] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:16.891070 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65842aa-63b6-468c-9885-7dcc618feb2a None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:16.891368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65842aa-63b6-468c-9885-7dcc618feb2a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:16.891651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65842aa-63b6-468c-9885-7dcc618feb2a None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:16.893295 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 34/71] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:16.894652 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65842aa-63b6-468c-9885-7dcc618feb2a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:16.914580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8ec22-f50d-4547-9226-084d87e64945 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:16.965477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c536a0d-e652-4e1f-96a2-9a1542582cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:16.965838 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c536a0d-e652-4e1f-96a2-9a1542582cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:16.966075 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c536a0d-e652-4e1f-96a2-9a1542582cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:16.966356 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c536a0d-e652-4e1f-96a2-9a1542582cd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:16.967124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c536a0d-e652-4e1f-96a2-9a1542582cd0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:16.967461 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c536a0d-e652-4e1f-96a2-9a1542582cd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:16.968929 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c536a0d-e652-4e1f-96a2-9a1542582cd0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:16.984539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 38/72] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:55:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:16.986072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c536a0d-e652-4e1f-96a2-9a1542582cd0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:17.029506 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65842aa-63b6-468c-9885-7dcc618feb2a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:17.068469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b7745-5641-40af-9fa4-050fd635dedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:17.069520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b7745-5641-40af-9fa4-050fd635dedf None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:17.069681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b7745-5641-40af-9fa4-050fd635dedf None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:17.069806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b7745-5641-40af-9fa4-050fd635dedf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:17.078729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:17.082573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b7745-5641-40af-9fa4-050fd635dedf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-215b7745-5641-40af-9fa4-050fd635dedf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:17.091026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:17.092123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:17.092903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:17.092903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:17.092903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:17.092903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:17.092903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:17.092903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:17.092903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:17.092903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:17.092903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:17.092903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:17.092903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:17.092903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:17.093512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 35/73] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:55:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:17.096702 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f54837-7e92-46a3-8b93-26fe1198959f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:17.097199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f54837-7e92-46a3-8b93-26fe1198959f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:17.097488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f54837-7e92-46a3-8b93-26fe1198959f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:17.107850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:17.128999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e9f54837-7e92-46a3-8b93-26fe1198959f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:17.130497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e9f54837-7e92-46a3-8b93-26fe1198959f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:17.143072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c536a0d-e652-4e1f-96a2-9a1542582cd0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:17.191091 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0330817-5c7d-4f24-b1c2-0f62ea0647a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:17.192129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0330817-5c7d-4f24-b1c2-0f62ea0647a9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:17.192423 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0330817-5c7d-4f24-b1c2-0f62ea0647a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:17.192707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0330817-5c7d-4f24-b1c2-0f62ea0647a9 None admin] PATH_INFO: `/v3/users/e430ce5b82254dcb8def7ae26a6c6e27` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:17.209349 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 36/74] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:17.209827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0330817-5c7d-4f24-b1c2-0f62ea0647a9 None admin] RBAC: Authorizing `identity:get_user(user_id=e430ce5b82254dcb8def7ae26a6c6e27)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:17.210639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f54837-7e92-46a3-8b93-26fe1198959f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:17.211307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0330817-5c7d-4f24-b1c2-0f62ea0647a9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:17.213108 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 39/75] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:16 2022] GET /identity/v3/users/e430ce5b82254dcb8def7ae26a6c6e27 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:17.217019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad17949-dc32-4029-8e94-a31f54d2ef1b None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:17.217019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad17949-dc32-4029-8e94-a31f54d2ef1b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:17.217019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad17949-dc32-4029-8e94-a31f54d2ef1b None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:17.217019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 40/76] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:17.218191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad17949-dc32-4029-8e94-a31f54d2ef1b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:17.350798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f54837-7e92-46a3-8b93-26fe1198959f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:17.370503 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad17949-dc32-4029-8e94-a31f54d2ef1b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:17.383567 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:17.390193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f45d6-3cb8-4587-96e7-7f547b01bd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:17.391086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f45d6-3cb8-4587-96e7-7f547b01bd2b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:17.391412 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f45d6-3cb8-4587-96e7-7f547b01bd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:17.391657 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f45d6-3cb8-4587-96e7-7f547b01bd2b None admin] PATH_INFO: `/v3/users/e430ce5b82254dcb8def7ae26a6c6e27` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:17.392947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f45d6-3cb8-4587-96e7-7f547b01bd2b None admin] RBAC: Authorizing `identity:get_user(user_id=e430ce5b82254dcb8def7ae26a6c6e27)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:17.394433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f45d6-3cb8-4587-96e7-7f547b01bd2b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:17.396151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 37/77] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:17 2022] GET /identity/v3/users/e430ce5b82254dcb8def7ae26a6c6e27 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:17.396955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f45d6-3cb8-4587-96e7-7f547b01bd2b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:17.406548 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:17.414150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a1d28-1275-4fe5-8c06-79819c263254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:17.415077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a1d28-1275-4fe5-8c06-79819c263254 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:17.415373 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a1d28-1275-4fe5-8c06-79819c263254 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:17.415649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a1d28-1275-4fe5-8c06-79819c263254 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:17.416469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a1d28-1275-4fe5-8c06-79819c263254 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:17.416771 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a1d28-1275-4fe5-8c06-79819c263254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:17.418896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a1d28-1275-4fe5-8c06-79819c263254 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:17.428364 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 41/78] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:17.429682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03388ba1-8860-4574-ae56-cd8584eefee6 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:17.430057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03388ba1-8860-4574-ae56-cd8584eefee6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:17.430346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03388ba1-8860-4574-ae56-cd8584eefee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:17.449320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-03388ba1-8860-4574-ae56-cd8584eefee6 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:17.451431 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-03388ba1-8860-4574-ae56-cd8584eefee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:17.516661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f45d6-3cb8-4587-96e7-7f547b01bd2b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:17.555128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e3200-e01b-4d3b-ad70-ed072d5f4260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:17.555801 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e3200-e01b-4d3b-ad70-ed072d5f4260 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:17.556113 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e3200-e01b-4d3b-ad70-ed072d5f4260 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:17.556394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e3200-e01b-4d3b-ad70-ed072d5f4260 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:17.566618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 42/79] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:17.568297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03388ba1-8860-4574-ae56-cd8584eefee6 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:17.652573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:17.660987 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e3200-e01b-4d3b-ad70-ed072d5f4260 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:17.663843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e3200-e01b-4d3b-ad70-ed072d5f4260 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:17.718861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03388ba1-8860-4574-ae56-cd8584eefee6 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:17.762947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:17.776458 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f606c-1091-44f6-b9ee-d947fafd88ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:17.777272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f606c-1091-44f6-b9ee-d947fafd88ab None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:17.778400 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f606c-1091-44f6-b9ee-d947fafd88ab None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:17.778400 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f606c-1091-44f6-b9ee-d947fafd88ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:17.789717 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 38/80] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:17.790848 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e3200-e01b-4d3b-ad70-ed072d5f4260 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:17.794405 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f606c-1091-44f6-b9ee-d947fafd88ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-bc5f606c-1091-44f6-b9ee-d947fafd88ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:17.807983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:17.809465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:17.810224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:17.810224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:17.810224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:17.810224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:17.810224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:17.810224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:17.810224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:17.810224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:17.810224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:17.810224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:17.810224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:17.810224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:17.810792 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 43/81] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:17.812315 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f606c-1091-44f6-b9ee-d947fafd88ab None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:17.920453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:17.925294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e3200-e01b-4d3b-ad70-ed072d5f4260 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:17.972361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f606c-1091-44f6-b9ee-d947fafd88ab None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:17.979199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfd919-cb9a-437c-888b-f47bd8e0b18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:17.979878 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfd919-cb9a-437c-888b-f47bd8e0b18d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:17.980213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfd919-cb9a-437c-888b-f47bd8e0b18d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:17.980495 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfd919-cb9a-437c-888b-f47bd8e0b18d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:18.015213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8630c5-268f-4ebe-b6ec-9955c7e5992e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:18.016140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8630c5-268f-4ebe-b6ec-9955c7e5992e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:18.016418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8630c5-268f-4ebe-b6ec-9955c7e5992e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:18.016693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8630c5-268f-4ebe-b6ec-9955c7e5992e None admin] PATH_INFO: `/v3/users/af6fa97a676b4e53bc4e27a552d09f11` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:18.017956 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8630c5-268f-4ebe-b6ec-9955c7e5992e None admin] RBAC: Authorizing `identity:get_user(user_id=af6fa97a676b4e53bc4e27a552d09f11)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:18.019462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8630c5-268f-4ebe-b6ec-9955c7e5992e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:18.021225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 44/82] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:17 2022] GET /identity/v3/users/af6fa97a676b4e53bc4e27a552d09f11 => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:18.025644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8630c5-268f-4ebe-b6ec-9955c7e5992e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:18.086482 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcfd919-cb9a-437c-888b-f47bd8e0b18d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:18.088565 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcfd919-cb9a-437c-888b-f47bd8e0b18d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:18.089704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:18.163643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8630c5-268f-4ebe-b6ec-9955c7e5992e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:18.204410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82d42c-d02e-4b00-bf23-a2f8da0f31d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:18.205171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82d42c-d02e-4b00-bf23-a2f8da0f31d4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:18.205458 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82d42c-d02e-4b00-bf23-a2f8da0f31d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:18.205741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82d42c-d02e-4b00-bf23-a2f8da0f31d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:18.207126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82d42c-d02e-4b00-bf23-a2f8da0f31d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:18.208726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82d42c-d02e-4b00-bf23-a2f8da0f31d4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:18.210512 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 45/83] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:55:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:18.211352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82d42c-d02e-4b00-bf23-a2f8da0f31d4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:18.246201 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:18.252706 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 39/84] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:18.254231 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfd919-cb9a-437c-888b-f47bd8e0b18d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:18.364469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82d42c-d02e-4b00-bf23-a2f8da0f31d4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:18.394252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:18.410252 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfd919-cb9a-437c-888b-f47bd8e0b18d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:18.416552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a3536-c9b1-4b57-b186-69b4d25d3eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:18.417558 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a3536-c9b1-4b57-b186-69b4d25d3eee None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:18.417928 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a3536-c9b1-4b57-b186-69b4d25d3eee None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:18.418578 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a3536-c9b1-4b57-b186-69b4d25d3eee None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:18.420155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a3536-c9b1-4b57-b186-69b4d25d3eee None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:18.421808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a3536-c9b1-4b57-b186-69b4d25d3eee None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:18.423757 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 46/85] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:18 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:18.424764 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a3536-c9b1-4b57-b186-69b4d25d3eee None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:18.460049 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f13327-9b64-4e35-b222-d96ea4be592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:18.460644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f13327-9b64-4e35-b222-d96ea4be592b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:18.460934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f13327-9b64-4e35-b222-d96ea4be592b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:18.461220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f13327-9b64-4e35-b222-d96ea4be592b None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:18.461990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f13327-9b64-4e35-b222-d96ea4be592b None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:18.462302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f13327-9b64-4e35-b222-d96ea4be592b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:18.463745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f13327-9b64-4e35-b222-d96ea4be592b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:18.474579 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 40/86] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:18.475430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f13327-9b64-4e35-b222-d96ea4be592b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:18.557446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a3536-c9b1-4b57-b186-69b4d25d3eee None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:18.561415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:18.615426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa63ec-1597-453a-9758-5ddb897de96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:18.616396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa63ec-1597-453a-9758-5ddb897de96c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:18.616807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa63ec-1597-453a-9758-5ddb897de96c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:18.617200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa63ec-1597-453a-9758-5ddb897de96c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:18.628999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f13327-9b64-4e35-b222-d96ea4be592b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:18.687060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd977a8b-a05d-4c36-a78a-92a9545b95cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:18.688171 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd977a8b-a05d-4c36-a78a-92a9545b95cc None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:18.688639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd977a8b-a05d-4c36-a78a-92a9545b95cc None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:18.689095 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd977a8b-a05d-4c36-a78a-92a9545b95cc None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:18.690098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd977a8b-a05d-4c36-a78a-92a9545b95cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:18.692613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd977a8b-a05d-4c36-a78a-92a9545b95cc None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:18.714298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:18.727462 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 41/87] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:55:18 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:55:18.728890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd977a8b-a05d-4c36-a78a-92a9545b95cc None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:18.737703 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa63ec-1597-453a-9758-5ddb897de96c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:18.740552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa63ec-1597-453a-9758-5ddb897de96c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:18.894333 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 47/88] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:18.895291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa63ec-1597-453a-9758-5ddb897de96c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:18.926830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd977a8b-a05d-4c36-a78a-92a9545b95cc None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:18.948575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:18.985703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000544de-d70c-416f-99b8-30cd1dcfb80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:18.987366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000544de-d70c-416f-99b8-30cd1dcfb80b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:18.987366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000544de-d70c-416f-99b8-30cd1dcfb80b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:18.987366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000544de-d70c-416f-99b8-30cd1dcfb80b None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:18.989004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000544de-d70c-416f-99b8-30cd1dcfb80b None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:18.989952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000544de-d70c-416f-99b8-30cd1dcfb80b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:18.991951 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 42/89] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:18 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:18.992595 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000544de-d70c-416f-99b8-30cd1dcfb80b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:19.064911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:19.107866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa63ec-1597-453a-9758-5ddb897de96c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:19.150911 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000544de-d70c-416f-99b8-30cd1dcfb80b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:19.186256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b6c4b-30c2-4d6a-ace6-806feca682c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:19.187042 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b6c4b-30c2-4d6a-ace6-806feca682c1 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:19.187344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b6c4b-30c2-4d6a-ace6-806feca682c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:19.187591 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b6c4b-30c2-4d6a-ace6-806feca682c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:19.188478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b6c4b-30c2-4d6a-ace6-806feca682c1 None admin] RBAC: Adding query filter params (role.id=3d7c80cf23864f58bf21f893ba7691d6, scope.project.id=e1099d961ac5441c973dfbd04812f431, user.id=e430ce5b82254dcb8def7ae26a6c6e27) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:19.188962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b6c4b-30c2-4d6a-ace6-806feca682c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:19.190823 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b6c4b-30c2-4d6a-ace6-806feca682c1 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:19.197496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 48/90] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:18 2022] GET /identity/v3/role_assignments?user.id=e430ce5b82254dcb8def7ae26a6c6e27&scope.project.id=e1099d961ac5441c973dfbd04812f431&role.id=3d7c80cf23864f58bf21f893ba7691d6 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:19.202665 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b6c4b-30c2-4d6a-ace6-806feca682c1 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:19.260874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8dc518-26c6-41a3-8351-3664e14b9081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:19.261681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dc518-26c6-41a3-8351-3664e14b9081 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:19.262580 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dc518-26c6-41a3-8351-3664e14b9081 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:19.263014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dc518-26c6-41a3-8351-3664e14b9081 None admin] PATH_INFO: `/v3/users/e430ce5b82254dcb8def7ae26a6c6e27` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:19.265665 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8dc518-26c6-41a3-8351-3664e14b9081 None admin] RBAC: Authorizing `identity:get_user(user_id=e430ce5b82254dcb8def7ae26a6c6e27)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:19.267184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8dc518-26c6-41a3-8351-3664e14b9081 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:19.269019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 43/91] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:18 2022] GET /identity/v3/users/e430ce5b82254dcb8def7ae26a6c6e27 => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:19.272147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8dc518-26c6-41a3-8351-3664e14b9081 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:19.352643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:19.412302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8dc518-26c6-41a3-8351-3664e14b9081 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:19.430049 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b6c4b-30c2-4d6a-ace6-806feca682c1 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:19.453474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbabf00-eecc-4dde-bd86-27490160ed3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:19.454164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbabf00-eecc-4dde-bd86-27490160ed3c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:19.454450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbabf00-eecc-4dde-bd86-27490160ed3c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:19.454736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbabf00-eecc-4dde-bd86-27490160ed3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:19.455791 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbabf00-eecc-4dde-bd86-27490160ed3c None admin] RBAC: Adding query filter params (role.id=5a989bb8c8a1465ca76622ffb504dee5, scope.project.id=e1099d961ac5441c973dfbd04812f431, user.id=e430ce5b82254dcb8def7ae26a6c6e27) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:19.455980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbabf00-eecc-4dde-bd86-27490160ed3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:19.457461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbabf00-eecc-4dde-bd86-27490160ed3c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:19.463802 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 44/92] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:19 2022] GET /identity/v3/role_assignments?user.id=e430ce5b82254dcb8def7ae26a6c6e27&scope.project.id=e1099d961ac5441c973dfbd04812f431&role.id=5a989bb8c8a1465ca76622ffb504dee5 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:19.466125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbabf00-eecc-4dde-bd86-27490160ed3c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:19.561535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88ee84-91cf-42c1-b0a7-a7b5eb22671e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:19.562319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88ee84-91cf-42c1-b0a7-a7b5eb22671e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:19.562612 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88ee84-91cf-42c1-b0a7-a7b5eb22671e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:19.562868 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88ee84-91cf-42c1-b0a7-a7b5eb22671e None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:19.568876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88ee84-91cf-42c1-b0a7-a7b5eb22671e None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:19.570425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88ee84-91cf-42c1-b0a7-a7b5eb22671e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:19.572314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 49/93] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:19 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:19.585288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88ee84-91cf-42c1-b0a7-a7b5eb22671e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:19.586257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbabf00-eecc-4dde-bd86-27490160ed3c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:19.626025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:19.627446 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:19.636451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9524d2a-f825-4964-8f8b-54e9b285d1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:19.637368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9524d2a-f825-4964-8f8b-54e9b285d1ab None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:19.637924 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9524d2a-f825-4964-8f8b-54e9b285d1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:19.638328 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9524d2a-f825-4964-8f8b-54e9b285d1ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:19.786516 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9524d2a-f825-4964-8f8b-54e9b285d1ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:19.788199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9524d2a-f825-4964-8f8b-54e9b285d1ab None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:19.860025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88ee84-91cf-42c1-b0a7-a7b5eb22671e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:19.926366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ea191-0c32-43e3-82ad-b877fbea6794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:19.928197 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ea191-0c32-43e3-82ad-b877fbea6794 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:19.928935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ea191-0c32-43e3-82ad-b877fbea6794 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:19.929647 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ea191-0c32-43e3-82ad-b877fbea6794 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:19.931825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ea191-0c32-43e3-82ad-b877fbea6794 None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=e1099d961ac5441c973dfbd04812f431, user.id=af6fa97a676b4e53bc4e27a552d09f11) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:19.932806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ea191-0c32-43e3-82ad-b877fbea6794 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:19.937395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ea191-0c32-43e3-82ad-b877fbea6794 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:19.948430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 50/94] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:19 2022] GET /identity/v3/role_assignments?user.id=af6fa97a676b4e53bc4e27a552d09f11&scope.project.id=e1099d961ac5441c973dfbd04812f431&role.id=000e942517f2469ba36d2e65978bc314 => generated 259 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:19.963640 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 45/95] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:19.973727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ea191-0c32-43e3-82ad-b877fbea6794 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:20.127825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:20.164374 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ea191-0c32-43e3-82ad-b877fbea6794 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:20.240088 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee80f75-43a7-4523-9468-29b604a75c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:20.240975 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee80f75-43a7-4523-9468-29b604a75c0a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:20.241262 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee80f75-43a7-4523-9468-29b604a75c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:20.241539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee80f75-43a7-4523-9468-29b604a75c0a None admin] PATH_INFO: `/v3/projects/a66f42db6df147daa57765d7d45ade8c` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:20.248782 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee80f75-43a7-4523-9468-29b604a75c0a None admin] RBAC: Authorizing `identity:get_project(project_id=a66f42db6df147daa57765d7d45ade8c)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:20.250302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee80f75-43a7-4523-9468-29b604a75c0a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:20.252533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 51/96] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:19 2022] GET /identity/v3/projects/a66f42db6df147daa57765d7d45ade8c => generated 312 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:20.261756 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9524d2a-f825-4964-8f8b-54e9b285d1ab None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:20.304259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:20.510377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9524d2a-f825-4964-8f8b-54e9b285d1ab None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:20.588075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c55ab5-feb6-41ba-b93f-8d2534ffdfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:20.589245 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c55ab5-feb6-41ba-b93f-8d2534ffdfda None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:20.589741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c55ab5-feb6-41ba-b93f-8d2534ffdfda None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:20.590795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c55ab5-feb6-41ba-b93f-8d2534ffdfda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:20.592538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c55ab5-feb6-41ba-b93f-8d2534ffdfda None admin] RBAC: Adding query filter params (role.id=3d7c80cf23864f58bf21f893ba7691d6, scope.project.id=a66f42db6df147daa57765d7d45ade8c, user.id=e430ce5b82254dcb8def7ae26a6c6e27) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:20.593119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c55ab5-feb6-41ba-b93f-8d2534ffdfda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:20.595832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c55ab5-feb6-41ba-b93f-8d2534ffdfda None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:20.611986 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 46/97] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:20 2022] GET /identity/v3/role_assignments?user.id=e430ce5b82254dcb8def7ae26a6c6e27&scope.project.id=a66f42db6df147daa57765d7d45ade8c&role.id=3d7c80cf23864f58bf21f893ba7691d6 => generated 259 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:22.683271 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc16f45-269e-4481-950a-1edc10107f1d None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:22.683895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc16f45-269e-4481-950a-1edc10107f1d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:22.684084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc16f45-269e-4481-950a-1edc10107f1d None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:22.685367 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 52/98] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:22.691258 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509c7c4-6fe3-4778-aec8-fba4be5d3b8c None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:22.691883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509c7c4-6fe3-4778-aec8-fba4be5d3b8c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:22.691883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509c7c4-6fe3-4778-aec8-fba4be5d3b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:22.724193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d509c7c4-6fe3-4778-aec8-fba4be5d3b8c None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:22.725413 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d509c7c4-6fe3-4778-aec8-fba4be5d3b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:22.827060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 47/99] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:55:22.882016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc16f45-269e-4481-950a-1edc10107f1d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:22.900949 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec616b7d-c29d-4c6c-93a5-1e7f709567b2 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:22.901606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec616b7d-c29d-4c6c-93a5-1e7f709567b2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:22.902077 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec616b7d-c29d-4c6c-93a5-1e7f709567b2 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:22.903792 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 48/100] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:55:22.909587 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10718c-9557-4700-b7bd-cad30e8e0121 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:22.910175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10718c-9557-4700-b7bd-cad30e8e0121 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:22.910676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10718c-9557-4700-b7bd-cad30e8e0121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:22.938765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4a10718c-9557-4700-b7bd-cad30e8e0121 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:22.941138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4a10718c-9557-4700-b7bd-cad30e8e0121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:22.948391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:23.081851 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 49/101] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:55:23.087523 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e75101-e207-404d-8da9-be2e3e79dd09 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:23.088651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e75101-e207-404d-8da9-be2e3e79dd09 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:23.090360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e75101-e207-404d-8da9-be2e3e79dd09 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:23.097856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 50/102] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:23 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:23.114747 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd07177-8bfa-42f6-89b1-f09c262146fc None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:23.115647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd07177-8bfa-42f6-89b1-f09c262146fc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:23.116460 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd07177-8bfa-42f6-89b1-f09c262146fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:23.149362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc16f45-269e-4481-950a-1edc10107f1d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:23.150903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7cd07177-8bfa-42f6-89b1-f09c262146fc None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:23.153502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7cd07177-8bfa-42f6-89b1-f09c262146fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:23.222594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf053bad-ec01-4255-b74c-c2a3585bbd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:23.224505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf053bad-ec01-4255-b74c-c2a3585bbd0b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:23.225331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf053bad-ec01-4255-b74c-c2a3585bbd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:23.226154 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf053bad-ec01-4255-b74c-c2a3585bbd0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:23.228963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf053bad-ec01-4255-b74c-c2a3585bbd0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:23.233351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf053bad-ec01-4255-b74c-c2a3585bbd0b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:23.237136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 53/103] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:55:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:23.239194 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf053bad-ec01-4255-b74c-c2a3585bbd0b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:23.283840 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:23.308391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 51/104] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:55:23.309620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd07177-8bfa-42f6-89b1-f09c262146fc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:23.416118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:23.430494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf053bad-ec01-4255-b74c-c2a3585bbd0b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:23.466717 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd07177-8bfa-42f6-89b1-f09c262146fc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:23.489901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d705730-3eaf-48e6-89d7-1edc72148f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:23.491172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d705730-3eaf-48e6-89d7-1edc72148f63 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:23.491699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d705730-3eaf-48e6-89d7-1edc72148f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:23.492710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d705730-3eaf-48e6-89d7-1edc72148f63 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:23.510132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32887877-8515-4eeb-a5ca-372c7fbd9e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:23.511021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32887877-8515-4eeb-a5ca-372c7fbd9e76 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:23.511473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32887877-8515-4eeb-a5ca-372c7fbd9e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:23.512496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32887877-8515-4eeb-a5ca-372c7fbd9e76 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:23.512850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d705730-3eaf-48e6-89d7-1edc72148f63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:23.513488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32887877-8515-4eeb-a5ca-372c7fbd9e76 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:23.515141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32887877-8515-4eeb-a5ca-372c7fbd9e76 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-3d705730-3eaf-48e6-89d7-1edc72148f63 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:23.529085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:23.530679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:23.533291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:23.533291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:23.533291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:23.533291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:23.533291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:23.533291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:23.533291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:23.533291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:23.533291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:23.533291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:23.533291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:23.533291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:23.533291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 54/105] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:23.534487 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb24db-f78d-4f8c-b084-60968769beae None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:23.534487 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb24db-f78d-4f8c-b084-60968769beae None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:23.534704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb24db-f78d-4f8c-b084-60968769beae None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:23.537218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 55/106] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:23.539260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046326a6-5efe-4154-af6e-b7eb2ffedf50 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:23.540084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046326a6-5efe-4154-af6e-b7eb2ffedf50 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:23.540587 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046326a6-5efe-4154-af6e-b7eb2ffedf50 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:23.550182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 56/107] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:23 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:23.551582 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046326a6-5efe-4154-af6e-b7eb2ffedf50 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:23.561421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 52/108] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jul 21 08:55:23 2022] POST /identity/v3/projects => generated 302 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:55:23.562568 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32887877-8515-4eeb-a5ca-372c7fbd9e76 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:23.745192 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:23.806479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046326a6-5efe-4154-af6e-b7eb2ffedf50 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:23.808037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32887877-8515-4eeb-a5ca-372c7fbd9e76 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:23.858296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:23.882812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce15477-26f8-4b15-8072-1633c3b7c9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:23.883694 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce15477-26f8-4b15-8072-1633c3b7c9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:23.884047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce15477-26f8-4b15-8072-1633c3b7c9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:23.884330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce15477-26f8-4b15-8072-1633c3b7c9f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:23.888724 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c90f0-aebc-4f60-829f-060703e3c09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:23.890397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c90f0-aebc-4f60-829f-060703e3c09d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:23.891208 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c90f0-aebc-4f60-829f-060703e3c09d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:23.892025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c90f0-aebc-4f60-829f-060703e3c09d None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:23.893739 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c90f0-aebc-4f60-829f-060703e3c09d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:23.894789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c90f0-aebc-4f60-829f-060703e3c09d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:23.898000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c90f0-aebc-4f60-829f-060703e3c09d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:23.908878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce15477-26f8-4b15-8072-1633c3b7c9f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:23.916147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 53/109] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:23.920320 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1b3b8-9ed7-4744-b0d0-38a221a38f3e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:23.920944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1b3b8-9ed7-4744-b0d0-38a221a38f3e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:23.921566 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1b3b8-9ed7-4744-b0d0-38a221a38f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-fce15477-26f8-4b15-8072-1633c3b7c9f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:23.928478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:23.930113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:23.931509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:23.931509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:23.931509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:23.931509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:23.931509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:23.931509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:23.931509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:23.931509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:23.931509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:23.931509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:23.931509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:55:23.931509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:23.935619 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 57/110] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:55:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:23.941360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94957b4-2ac9-4ed2-97b2-bb1c17d973f5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:23.941360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94957b4-2ac9-4ed2-97b2-bb1c17d973f5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:23.941360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94957b4-2ac9-4ed2-97b2-bb1c17d973f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:23.960741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0af1b3b8-9ed7-4744-b0d0-38a221a38f3e None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:23.962867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0af1b3b8-9ed7-4744-b0d0-38a221a38f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:23.981796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f94957b4-2ac9-4ed2-97b2-bb1c17d973f5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:23.984998 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f94957b4-2ac9-4ed2-97b2-bb1c17d973f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:24.103536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 54/111] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:24.107785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1b3b8-9ed7-4744-b0d0-38a221a38f3e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:24.147414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 58/112] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:24.156759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94957b4-2ac9-4ed2-97b2-bb1c17d973f5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:24.304159 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:24.350528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1b3b8-9ed7-4744-b0d0-38a221a38f3e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:24.360878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94957b4-2ac9-4ed2-97b2-bb1c17d973f5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:24.379401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:24.419007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218027ad-2c54-4afc-b86f-5ccf7f5f66c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:24.420712 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218027ad-2c54-4afc-b86f-5ccf7f5f66c8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:24.421239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218027ad-2c54-4afc-b86f-5ccf7f5f66c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:24.421783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218027ad-2c54-4afc-b86f-5ccf7f5f66c8 None admin] PATH_INFO: `/v3/users/7ba631c71055479f9cadb3aeaf8d8d68` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:24.424971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0eb8c-088d-497f-b333-c1b33e7f4d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:24.426118 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0eb8c-088d-497f-b333-c1b33e7f4d04 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:24.426636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0eb8c-088d-497f-b333-c1b33e7f4d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:24.427722 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0eb8c-088d-497f-b333-c1b33e7f4d04 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:24.428677 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0eb8c-088d-497f-b333-c1b33e7f4d04 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:24.429245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0eb8c-088d-497f-b333-c1b33e7f4d04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:24.431990 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0eb8c-088d-497f-b333-c1b33e7f4d04 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:24.449958 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 59/113] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:55:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:24.452196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0eb8c-088d-497f-b333-c1b33e7f4d04 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:24.459785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218027ad-2c54-4afc-b86f-5ccf7f5f66c8 None admin] RBAC: Authorizing `identity:get_user(user_id=7ba631c71055479f9cadb3aeaf8d8d68)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:24.462738 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218027ad-2c54-4afc-b86f-5ccf7f5f66c8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:24.470996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 55/114] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:24 2022] GET /identity/v3/users/7ba631c71055479f9cadb3aeaf8d8d68 => generated 289 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:24.470996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218027ad-2c54-4afc-b86f-5ccf7f5f66c8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:24.650692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0eb8c-088d-497f-b333-c1b33e7f4d04 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:24.666028 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218027ad-2c54-4afc-b86f-5ccf7f5f66c8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:24.718836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e910f-6208-4012-b5af-5ecb34603c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:24.720194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e910f-6208-4012-b5af-5ecb34603c02 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:24.720671 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e910f-6208-4012-b5af-5ecb34603c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:24.721136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e910f-6208-4012-b5af-5ecb34603c02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:24.722004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:24.733445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acaa5a-8d5d-420d-9104-6e47e194c049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:24.734850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acaa5a-8d5d-420d-9104-6e47e194c049 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:24.735435 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acaa5a-8d5d-420d-9104-6e47e194c049 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:24.735956 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16acaa5a-8d5d-420d-9104-6e47e194c049 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:24.741323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e910f-6208-4012-b5af-5ecb34603c02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:24.752521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16acaa5a-8d5d-420d-9104-6e47e194c049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-635e910f-6208-4012-b5af-5ecb34603c02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:24.758551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:24.760193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:24.763118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:24.763118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:24.763118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:24.763118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:24.763118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:24.763118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:24.763118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:24.763118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:24.763118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:24.763118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:24.763118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:24.763118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:24.763118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 56/115] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:24.765050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00ea2d-74fb-4080-a51a-a84375bb0f65 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:24.765050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00ea2d-74fb-4080-a51a-a84375bb0f65 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:24.765050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00ea2d-74fb-4080-a51a-a84375bb0f65 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:24.772397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 57/116] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:24.772397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00ea2d-74fb-4080-a51a-a84375bb0f65 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-16acaa5a-8d5d-420d-9104-6e47e194c049 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:24.772765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:24.774288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:24.780642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:24.780642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:24.780642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:24.780642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:24.780642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:24.780642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:24.780642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:24.780642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:24.780642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:24.780642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:24.780642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:24.780642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:24.780642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 60/117] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:55:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:24.780642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acaa5a-8d5d-420d-9104-6e47e194c049 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:24.841152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:24.958761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00ea2d-74fb-4080-a51a-a84375bb0f65 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:24.973834 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16acaa5a-8d5d-420d-9104-6e47e194c049 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:25.017602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e221d-f7f3-4d98-ab4b-924735431bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:25.019419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e221d-f7f3-4d98-ab4b-924735431bee None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:25.020204 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e221d-f7f3-4d98-ab4b-924735431bee None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:25.020933 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e221d-f7f3-4d98-ab4b-924735431bee None admin] PATH_INFO: `/v3/users/e430ce5b82254dcb8def7ae26a6c6e27` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:25.024496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e221d-f7f3-4d98-ab4b-924735431bee None admin] RBAC: Authorizing `identity:get_user(user_id=e430ce5b82254dcb8def7ae26a6c6e27)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:25.028338 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e221d-f7f3-4d98-ab4b-924735431bee None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:25.033260 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 58/118] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:24 2022] GET /identity/v3/users/e430ce5b82254dcb8def7ae26a6c6e27 => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:25.035479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e221d-f7f3-4d98-ab4b-924735431bee None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:25.043966 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ba446-977a-4592-a082-4321a01faa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:25.045071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ba446-977a-4592-a082-4321a01faa05 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:25.045562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ba446-977a-4592-a082-4321a01faa05 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:25.046045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ba446-977a-4592-a082-4321a01faa05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:25.164259 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:25.185460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ba446-977a-4592-a082-4321a01faa05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:25.187379 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ba446-977a-4592-a082-4321a01faa05 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:25.188904 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:25.192685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e221d-f7f3-4d98-ab4b-924735431bee None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:25.245714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf4a24-d448-4051-a6f5-60e618e2dac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:25.247607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf4a24-d448-4051-a6f5-60e618e2dac0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:25.248115 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf4a24-d448-4051-a6f5-60e618e2dac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:25.248573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf4a24-d448-4051-a6f5-60e618e2dac0 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:25.249913 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf4a24-d448-4051-a6f5-60e618e2dac0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:25.250417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf4a24-d448-4051-a6f5-60e618e2dac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:25.253574 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bf4a24-d448-4051-a6f5-60e618e2dac0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:25.269787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 59/119] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:25.271630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125ffd5-2dff-4468-92bc-d59495cc21ca None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:25.272178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125ffd5-2dff-4468-92bc-d59495cc21ca None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:25.272680 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125ffd5-2dff-4468-92bc-d59495cc21ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:25.296222 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-8125ffd5-2dff-4468-92bc-d59495cc21ca None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:25.298129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8125ffd5-2dff-4468-92bc-d59495cc21ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:25.370185 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 61/120] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:25.371661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ba446-977a-4592-a082-4321a01faa05 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:25.401175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 60/121] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:25.402081 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125ffd5-2dff-4468-92bc-d59495cc21ca None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:25.506511 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:25.522016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ba446-977a-4592-a082-4321a01faa05 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:25.537186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:25.553335 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125ffd5-2dff-4468-92bc-d59495cc21ca None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:25.585915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3213805-1c42-45ed-979d-112c2176891c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:25.586812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3213805-1c42-45ed-979d-112c2176891c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:25.587278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3213805-1c42-45ed-979d-112c2176891c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:25.587736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3213805-1c42-45ed-979d-112c2176891c None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:25.589115 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3213805-1c42-45ed-979d-112c2176891c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:25.589677 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3213805-1c42-45ed-979d-112c2176891c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:25.592055 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3213805-1c42-45ed-979d-112c2176891c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:25.611899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 62/122] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:25.611899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3213805-1c42-45ed-979d-112c2176891c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:25.622336 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4654a8-be64-4d5b-9ad9-c1a924242fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:25.623878 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4654a8-be64-4d5b-9ad9-c1a924242fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:25.624465 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4654a8-be64-4d5b-9ad9-c1a924242fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:25.624899 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4654a8-be64-4d5b-9ad9-c1a924242fb4 None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:25.627108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4654a8-be64-4d5b-9ad9-c1a924242fb4 None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:25.631575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4654a8-be64-4d5b-9ad9-c1a924242fb4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:25.632396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 61/123] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:25 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:25.633829 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4654a8-be64-4d5b-9ad9-c1a924242fb4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:25.778479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3213805-1c42-45ed-979d-112c2176891c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:25.778997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4654a8-be64-4d5b-9ad9-c1a924242fb4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:25.823731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f08715-8597-4112-b301-925ceebcc426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:25.824112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5ab20-8b76-407c-a220-4d94e688090f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:25.824770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5ab20-8b76-407c-a220-4d94e688090f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:25.825074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5ab20-8b76-407c-a220-4d94e688090f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:25.825357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5ab20-8b76-407c-a220-4d94e688090f None admin] PATH_INFO: `/v3/users/e430ce5b82254dcb8def7ae26a6c6e27` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:25.825872 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f08715-8597-4112-b301-925ceebcc426 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:25.826422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f08715-8597-4112-b301-925ceebcc426 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:25.826828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5ab20-8b76-407c-a220-4d94e688090f None admin] RBAC: Authorizing `identity:get_user(user_id=e430ce5b82254dcb8def7ae26a6c6e27)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:25.826984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f08715-8597-4112-b301-925ceebcc426 None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:25.828570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5ab20-8b76-407c-a220-4d94e688090f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:25.828739 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f08715-8597-4112-b301-925ceebcc426 None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:25.834924 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 63/124] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:25 2022] GET /identity/v3/users/e430ce5b82254dcb8def7ae26a6c6e27 => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:25.834924 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5ab20-8b76-407c-a220-4d94e688090f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:25.835202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f08715-8597-4112-b301-925ceebcc426 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:25.835202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 62/125] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:25 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:25.835202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f08715-8597-4112-b301-925ceebcc426 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:25.870584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:25.884719 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:25.977064 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f08715-8597-4112-b301-925ceebcc426 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:25.982810 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5ab20-8b76-407c-a220-4d94e688090f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:26.019725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71458835-7615-4b38-8577-73a80781dcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:26.021576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71458835-7615-4b38-8577-73a80781dcb5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:26.022217 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71458835-7615-4b38-8577-73a80781dcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:26.022801 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71458835-7615-4b38-8577-73a80781dcb5 None admin] PATH_INFO: `/v3/users/af6fa97a676b4e53bc4e27a552d09f11` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:26.025130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71458835-7615-4b38-8577-73a80781dcb5 None admin] RBAC: Authorizing `identity:get_user(user_id=af6fa97a676b4e53bc4e27a552d09f11)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:26.028654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71458835-7615-4b38-8577-73a80781dcb5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:26.031831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 63/126] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:25 2022] GET /identity/v3/users/af6fa97a676b4e53bc4e27a552d09f11 => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:26.033355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71458835-7615-4b38-8577-73a80781dcb5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:26.039244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcf489-50d1-4fd6-ad15-3e49df3b3e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:26.040516 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcf489-50d1-4fd6-ad15-3e49df3b3e2d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:26.040937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcf489-50d1-4fd6-ad15-3e49df3b3e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:26.041356 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcf489-50d1-4fd6-ad15-3e49df3b3e2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:26.055005 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcf489-50d1-4fd6-ad15-3e49df3b3e2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-c6bcf489-50d1-4fd6-ad15-3e49df3b3e2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:26.063585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:26.065240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:26.066671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:26.066671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:26.066671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:26.066671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:26.066671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:26.066671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:26.066671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:26.066671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:26.066671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:26.066671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:26.066671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:26.066671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:26.070033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 64/127] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:26.071401 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcf489-50d1-4fd6-ad15-3e49df3b3e2d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:26.206106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:26.213195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71458835-7615-4b38-8577-73a80781dcb5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:26.215690 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:26.247581 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcf489-50d1-4fd6-ad15-3e49df3b3e2d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:26.253647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe69478-5121-475a-9efb-8bd9cc23711b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:26.254673 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe69478-5121-475a-9efb-8bd9cc23711b None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:26.255043 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe69478-5121-475a-9efb-8bd9cc23711b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:26.255421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe69478-5121-475a-9efb-8bd9cc23711b None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431/users/e430ce5b82254dcb8def7ae26a6c6e27/roles/5a989bb8c8a1465ca76622ffb504dee5` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:26.258372 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe69478-5121-475a-9efb-8bd9cc23711b None admin] RBAC: Authorizing `identity:create_grant(project_id=e1099d961ac5441c973dfbd04812f431, user_id=e430ce5b82254dcb8def7ae26a6c6e27, role_id=5a989bb8c8a1465ca76622ffb504dee5)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:26.260203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe69478-5121-475a-9efb-8bd9cc23711b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:26.276444 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 64/128] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:55:26 2022] PUT /identity/v3/projects/e1099d961ac5441c973dfbd04812f431/users/e430ce5b82254dcb8def7ae26a6c6e27/roles/5a989bb8c8a1465ca76622ffb504dee5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:55:26.277247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe69478-5121-475a-9efb-8bd9cc23711b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:26.303027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e24f74-1e9b-4ff7-9749-1a99f2de240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:26.304389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e24f74-1e9b-4ff7-9749-1a99f2de240b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:26.304839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e24f74-1e9b-4ff7-9749-1a99f2de240b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:26.305281 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e24f74-1e9b-4ff7-9749-1a99f2de240b None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:26.307598 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e24f74-1e9b-4ff7-9749-1a99f2de240b None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:26.310266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e24f74-1e9b-4ff7-9749-1a99f2de240b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:26.313709 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 65/129] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:26 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:26.314904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e24f74-1e9b-4ff7-9749-1a99f2de240b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:26.464688 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e24f74-1e9b-4ff7-9749-1a99f2de240b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:26.478631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe69478-5121-475a-9efb-8bd9cc23711b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:26.506425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4ca65-9d87-45ea-be3d-57236b757a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:26.507306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4ca65-9d87-45ea-be3d-57236b757a39 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:26.507587 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4ca65-9d87-45ea-be3d-57236b757a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:26.507871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4ca65-9d87-45ea-be3d-57236b757a39 None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:26.509168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4ca65-9d87-45ea-be3d-57236b757a39 None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:26.510943 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4ca65-9d87-45ea-be3d-57236b757a39 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:26.512666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 66/130] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:26 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:26.513399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4ca65-9d87-45ea-be3d-57236b757a39 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:26.529950 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7dc6e-8764-41b6-ae37-15ead8a3e301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:26.531054 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7dc6e-8764-41b6-ae37-15ead8a3e301 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:26.531572 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7dc6e-8764-41b6-ae37-15ead8a3e301 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:26.531947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7dc6e-8764-41b6-ae37-15ead8a3e301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:26.532884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7dc6e-8764-41b6-ae37-15ead8a3e301 None admin] RBAC: Adding query filter params (role.id=0a6e69b6ee9e43659f71a5ba3260f5c9, scope.project.id=e1099d961ac5441c973dfbd04812f431, user.id=7ba631c71055479f9cadb3aeaf8d8d68) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:26.533207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7dc6e-8764-41b6-ae37-15ead8a3e301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:26.534694 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7dc6e-8764-41b6-ae37-15ead8a3e301 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:26.535148 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:26.542004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 65/131] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:26 2022] GET /identity/v3/role_assignments?user.id=7ba631c71055479f9cadb3aeaf8d8d68&scope.project.id=e1099d961ac5441c973dfbd04812f431&role.id=0a6e69b6ee9e43659f71a5ba3260f5c9 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:26.543722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a98f7-606f-4888-9542-055f7568024e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:26.544248 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a98f7-606f-4888-9542-055f7568024e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:26.544538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a98f7-606f-4888-9542-055f7568024e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:26.546225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 66/132] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:26.551118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a98f7-606f-4888-9542-055f7568024e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:26.602730 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:26.659373 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4ca65-9d87-45ea-be3d-57236b757a39 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:26.704965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24eb547-e1d7-460d-a8b5-7c03fd6450ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:26.705713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24eb547-e1d7-460d-a8b5-7c03fd6450ae None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:26.706122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24eb547-e1d7-460d-a8b5-7c03fd6450ae None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:26.706414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24eb547-e1d7-460d-a8b5-7c03fd6450ae None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:26.707201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24eb547-e1d7-460d-a8b5-7c03fd6450ae None admin] RBAC: Adding query filter params (name=member) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:26.707514 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24eb547-e1d7-460d-a8b5-7c03fd6450ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:26.708996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24eb547-e1d7-460d-a8b5-7c03fd6450ae None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:26.720228 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 67/133] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:26.720946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24eb547-e1d7-460d-a8b5-7c03fd6450ae None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:26.812396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a98f7-606f-4888-9542-055f7568024e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:26.856455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:26.885732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e954e5b-ec5c-4f47-ac78-a0fda77352de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:26.889559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e954e5b-ec5c-4f47-ac78-a0fda77352de None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:26.890335 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e954e5b-ec5c-4f47-ac78-a0fda77352de None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:26.891212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e954e5b-ec5c-4f47-ac78-a0fda77352de None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431/users/e430ce5b82254dcb8def7ae26a6c6e27/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:26.895690 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e954e5b-ec5c-4f47-ac78-a0fda77352de None admin] RBAC: Authorizing `identity:create_grant(project_id=e1099d961ac5441c973dfbd04812f431, user_id=e430ce5b82254dcb8def7ae26a6c6e27, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:26.897193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24eb547-e1d7-460d-a8b5-7c03fd6450ae None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:26.898911 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e954e5b-ec5c-4f47-ac78-a0fda77352de None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:26.930107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 67/134] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:55:26 2022] PUT /identity/v3/projects/e1099d961ac5441c973dfbd04812f431/users/e430ce5b82254dcb8def7ae26a6c6e27/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:55:26.932845 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09081443-5a35-4ad6-bb9c-bf42f7c1b8b1 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:26.933896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09081443-5a35-4ad6-bb9c-bf42f7c1b8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:26.934610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09081443-5a35-4ad6-bb9c-bf42f7c1b8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:26.964257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-09081443-5a35-4ad6-bb9c-bf42f7c1b8b1 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:26.968638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-09081443-5a35-4ad6-bb9c-bf42f7c1b8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:27.003889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9231329-ddce-4b04-9984-8b1422b0f8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:27.005533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9231329-ddce-4b04-9984-8b1422b0f8b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:27.007006 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9231329-ddce-4b04-9984-8b1422b0f8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:27.007698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9231329-ddce-4b04-9984-8b1422b0f8b7 None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431/users/af6fa97a676b4e53bc4e27a552d09f11/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:27.013570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9231329-ddce-4b04-9984-8b1422b0f8b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1099d961ac5441c973dfbd04812f431, user_id=af6fa97a676b4e53bc4e27a552d09f11, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:27.017537 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9231329-ddce-4b04-9984-8b1422b0f8b7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:27.049212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 68/135] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:55:26 2022] PUT /identity/v3/projects/e1099d961ac5441c973dfbd04812f431/users/af6fa97a676b4e53bc4e27a552d09f11/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:55:27.057221 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9231329-ddce-4b04-9984-8b1422b0f8b7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:27.111593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 68/136] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:27.228580 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09081443-5a35-4ad6-bb9c-bf42f7c1b8b1 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:27.270761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:27.326455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9231329-ddce-4b04-9984-8b1422b0f8b7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:27.413054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37af122-8372-4ef0-9ec7-0729f95afaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:27.414464 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37af122-8372-4ef0-9ec7-0729f95afaac None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:27.415178 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37af122-8372-4ef0-9ec7-0729f95afaac None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:27.415648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37af122-8372-4ef0-9ec7-0729f95afaac None admin] PATH_INFO: `/v3/users/e430ce5b82254dcb8def7ae26a6c6e27` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:27.423216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37af122-8372-4ef0-9ec7-0729f95afaac None admin] RBAC: Authorizing `identity:get_user(user_id=e430ce5b82254dcb8def7ae26a6c6e27)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:27.424820 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37af122-8372-4ef0-9ec7-0729f95afaac None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:27.426666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 69/137] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:27 2022] GET /identity/v3/users/e430ce5b82254dcb8def7ae26a6c6e27 => generated 275 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:27.436327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37af122-8372-4ef0-9ec7-0729f95afaac None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:27.452477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:27.512652 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09081443-5a35-4ad6-bb9c-bf42f7c1b8b1 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:27.584051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8f3ce-7149-458d-848f-d97aa2a2357c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:27.585419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8f3ce-7149-458d-848f-d97aa2a2357c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:27.585908 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37af122-8372-4ef0-9ec7-0729f95afaac None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:27.586062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8f3ce-7149-458d-848f-d97aa2a2357c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:27.586384 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8f3ce-7149-458d-848f-d97aa2a2357c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:27.588811 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8f3ce-7149-458d-848f-d97aa2a2357c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:27.591606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b8f3ce-7149-458d-848f-d97aa2a2357c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:27.595093 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 69/138] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:55:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:27.604873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8f3ce-7149-458d-848f-d97aa2a2357c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:27.643816 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1942139-894a-44d5-9686-aae659e3470c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:27.645736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1942139-894a-44d5-9686-aae659e3470c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:27.646199 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1942139-894a-44d5-9686-aae659e3470c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:27.646698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1942139-894a-44d5-9686-aae659e3470c None admin] PATH_INFO: `/v3/projects/a66f42db6df147daa57765d7d45ade8c` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:27.651702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1942139-894a-44d5-9686-aae659e3470c None admin] RBAC: Authorizing `identity:get_project(project_id=a66f42db6df147daa57765d7d45ade8c)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:27.658030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1942139-894a-44d5-9686-aae659e3470c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:27.664328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 70/139] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:27 2022] GET /identity/v3/projects/a66f42db6df147daa57765d7d45ade8c => generated 312 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:27.682142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1942139-894a-44d5-9686-aae659e3470c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:27.892515 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:27.943856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8f3ce-7149-458d-848f-d97aa2a2357c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:27.976650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:28.038739 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebb8ef-0346-46e0-8bb3-01f36843cfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:28.040040 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebb8ef-0346-46e0-8bb3-01f36843cfcb None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:28.040340 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebb8ef-0346-46e0-8bb3-01f36843cfcb None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:28.040625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebb8ef-0346-46e0-8bb3-01f36843cfcb None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:28.041383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebb8ef-0346-46e0-8bb3-01f36843cfcb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:28.043570 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebb8ef-0346-46e0-8bb3-01f36843cfcb None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:28.067003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1942139-894a-44d5-9686-aae659e3470c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:28.090074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 70/140] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:55:27 2022] POST /identity/v3/users => generated 283 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:55:28.165285 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4af8c-1db8-4f6b-94f4-11bee1969ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:28.166989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4af8c-1db8-4f6b-94f4-11bee1969ed4 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:28.167435 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4af8c-1db8-4f6b-94f4-11bee1969ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:28.168085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4af8c-1db8-4f6b-94f4-11bee1969ed4 None admin] PATH_INFO: `/v3/projects/a66f42db6df147daa57765d7d45ade8c/users/e430ce5b82254dcb8def7ae26a6c6e27/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:28.171708 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4af8c-1db8-4f6b-94f4-11bee1969ed4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a66f42db6df147daa57765d7d45ade8c, user_id=e430ce5b82254dcb8def7ae26a6c6e27, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:28.174290 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4af8c-1db8-4f6b-94f4-11bee1969ed4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:28.209004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 71/141] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:55:27 2022] PUT /identity/v3/projects/a66f42db6df147daa57765d7d45ade8c/users/e430ce5b82254dcb8def7ae26a6c6e27/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:55:29.800779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e11cb-e6e9-4b52-ba1e-79e8a430aebb None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:29.801553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e11cb-e6e9-4b52-ba1e-79e8a430aebb None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:29.801742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e11cb-e6e9-4b52-ba1e-79e8a430aebb None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:29.804488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 71/142] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:29.826007 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bfe99-2aeb-430c-bfa6-0f4c43489a29 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:29.827245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bfe99-2aeb-430c-bfa6-0f4c43489a29 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:29.827820 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bfe99-2aeb-430c-bfa6-0f4c43489a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:29.854234 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d88bfe99-2aeb-430c-bfa6-0f4c43489a29 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:29.857053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d88bfe99-2aeb-430c-bfa6-0f4c43489a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:29.968829 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 72/143] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:30.016500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986eba3-b94d-498d-a5fe-202d86106e7c None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:30.016794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986eba3-b94d-498d-a5fe-202d86106e7c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:30.017087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986eba3-b94d-498d-a5fe-202d86106e7c None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:30.017363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bfe99-2aeb-430c-bfa6-0f4c43489a29 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:30.018508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 72/144] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:30.026133 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29dd52e-dfb6-43b2-b35c-567ebcd5bc16 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:30.026429 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29dd52e-dfb6-43b2-b35c-567ebcd5bc16 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:30.026716 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29dd52e-dfb6-43b2-b35c-567ebcd5bc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:30.053963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a29dd52e-dfb6-43b2-b35c-567ebcd5bc16 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:30.055196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a29dd52e-dfb6-43b2-b35c-567ebcd5bc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:30.177911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:30.236940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 73/145] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:55:30.280662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bfe99-2aeb-430c-bfa6-0f4c43489a29 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:30.284121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29dd52e-dfb6-43b2-b35c-567ebcd5bc16 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:30.392719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca344c7-ef94-4b81-9c65-493be85f213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:30.393960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca344c7-ef94-4b81-9c65-493be85f213b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:30.394435 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca344c7-ef94-4b81-9c65-493be85f213b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:30.394941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca344c7-ef94-4b81-9c65-493be85f213b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:30.420113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca344c7-ef94-4b81-9c65-493be85f213b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:30.432207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-9ca344c7-ef94-4b81-9c65-493be85f213b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:30.435413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:30.436328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:30.437092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:30.437092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:30.437092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:30.437092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:30.437092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:30.437092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:30.437092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:30.437092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:30.437092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:30.437092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:30.437092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:30.437092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:30.439752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 73/146] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:30.447318 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca344c7-ef94-4b81-9c65-493be85f213b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:30.564908 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29dd52e-dfb6-43b2-b35c-567ebcd5bc16 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:30.681366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749b856-5dbd-430c-bd6a-8d71a2955515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:30.682201 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749b856-5dbd-430c-bd6a-8d71a2955515 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:30.682492 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749b856-5dbd-430c-bd6a-8d71a2955515 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:30.682773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749b856-5dbd-430c-bd6a-8d71a2955515 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:30.695643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca344c7-ef94-4b81-9c65-493be85f213b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:30.724889 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749b856-5dbd-430c-bd6a-8d71a2955515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:30.725026 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-2749b856-5dbd-430c-bd6a-8d71a2955515 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:30.733880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:30.734833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:30.735608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:30.735608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:30.735608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:30.735608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:30.735608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:30.735608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:30.735608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:30.735608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:30.735608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:30.735608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:30.735608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:55:30.735608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:30.738174 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 74/147] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:55:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 455 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:30.743946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83ae12-cf19-44d4-b9fc-2c3f2b60f556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:30.743946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83ae12-cf19-44d4-b9fc-2c3f2b60f556 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:30.743946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83ae12-cf19-44d4-b9fc-2c3f2b60f556 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:30.743946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83ae12-cf19-44d4-b9fc-2c3f2b60f556 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:30.744362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83ae12-cf19-44d4-b9fc-2c3f2b60f556 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:30.744362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83ae12-cf19-44d4-b9fc-2c3f2b60f556 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:30.744362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83ae12-cf19-44d4-b9fc-2c3f2b60f556 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:30.748071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749b856-5dbd-430c-bd6a-8d71a2955515 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:30.753832 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 74/148] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:30.759675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83ae12-cf19-44d4-b9fc-2c3f2b60f556 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:30.916785 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83ae12-cf19-44d4-b9fc-2c3f2b60f556 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:30.988124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041adb7d-43fc-4788-95a4-c8e2f99f6c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:30.990112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041adb7d-43fc-4788-95a4-c8e2f99f6c3d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:30.990799 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041adb7d-43fc-4788-95a4-c8e2f99f6c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:30.991615 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041adb7d-43fc-4788-95a4-c8e2f99f6c3d None admin] PATH_INFO: `/v3/users/7ba631c71055479f9cadb3aeaf8d8d68` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:30.994340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041adb7d-43fc-4788-95a4-c8e2f99f6c3d None admin] RBAC: Authorizing `identity:get_user(user_id=7ba631c71055479f9cadb3aeaf8d8d68)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:30.997915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041adb7d-43fc-4788-95a4-c8e2f99f6c3d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:31.001552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 75/149] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:30 2022] GET /identity/v3/users/7ba631c71055479f9cadb3aeaf8d8d68 => generated 289 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:31.011071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041adb7d-43fc-4788-95a4-c8e2f99f6c3d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:31.024414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749b856-5dbd-430c-bd6a-8d71a2955515 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:31.106952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:31.132666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:31.132903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa953350-da88-4643-84ab-d9a415db4b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:31.133522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa953350-da88-4643-84ab-d9a415db4b73 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:31.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa953350-da88-4643-84ab-d9a415db4b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:31.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa953350-da88-4643-84ab-d9a415db4b73 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:31.135052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa953350-da88-4643-84ab-d9a415db4b73 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:31.135442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa953350-da88-4643-84ab-d9a415db4b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:31.138124 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa953350-da88-4643-84ab-d9a415db4b73 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:31.155480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 75/150] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:55:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:31.193666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa953350-da88-4643-84ab-d9a415db4b73 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:31.205414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041adb7d-43fc-4788-95a4-c8e2f99f6c3d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:31.250839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c8863-210d-43d0-bc24-336e01ce3b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:31.252809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c8863-210d-43d0-bc24-336e01ce3b6c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:31.253554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c8863-210d-43d0-bc24-336e01ce3b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:31.254305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c8863-210d-43d0-bc24-336e01ce3b6c None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:31.256942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c8863-210d-43d0-bc24-336e01ce3b6c None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:31.260243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c8863-210d-43d0-bc24-336e01ce3b6c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:31.264515 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 76/151] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:31 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:31.275142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c8863-210d-43d0-bc24-336e01ce3b6c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:31.463719 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa953350-da88-4643-84ab-d9a415db4b73 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:31.523530 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c8863-210d-43d0-bc24-336e01ce3b6c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:31.562669 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945f34a-16cd-475e-95c3-8483f16a4c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:31.562669 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945f34a-16cd-475e-95c3-8483f16a4c82 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:31.562669 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945f34a-16cd-475e-95c3-8483f16a4c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:31.562669 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945f34a-16cd-475e-95c3-8483f16a4c82 None admin] PATH_INFO: `/v3/users/e430ce5b82254dcb8def7ae26a6c6e27` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:31.563357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945f34a-16cd-475e-95c3-8483f16a4c82 None admin] RBAC: Authorizing `identity:get_user(user_id=e430ce5b82254dcb8def7ae26a6c6e27)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:31.564914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945f34a-16cd-475e-95c3-8483f16a4c82 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:31.566900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 76/152] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:31 2022] GET /identity/v3/users/e430ce5b82254dcb8def7ae26a6c6e27 => generated 275 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:31.575962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945f34a-16cd-475e-95c3-8483f16a4c82 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:31.595647 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:31.644749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8921ae-1acf-4abc-b11a-93033077eb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:31.645674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8921ae-1acf-4abc-b11a-93033077eb62 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:31.645964 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8921ae-1acf-4abc-b11a-93033077eb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:31.646246 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8921ae-1acf-4abc-b11a-93033077eb62 None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431/users/7ba631c71055479f9cadb3aeaf8d8d68/roles/0a6e69b6ee9e43659f71a5ba3260f5c9` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:31.656905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8921ae-1acf-4abc-b11a-93033077eb62 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1099d961ac5441c973dfbd04812f431, user_id=7ba631c71055479f9cadb3aeaf8d8d68, role_id=0a6e69b6ee9e43659f71a5ba3260f5c9)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:31.658442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8921ae-1acf-4abc-b11a-93033077eb62 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:31.697583 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 77/153] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:55:31 2022] PUT /identity/v3/projects/e1099d961ac5441c973dfbd04812f431/users/7ba631c71055479f9cadb3aeaf8d8d68/roles/0a6e69b6ee9e43659f71a5ba3260f5c9 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:55:31.724464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:31.815782 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945f34a-16cd-475e-95c3-8483f16a4c82 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:31.864544 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c371a3-e5f5-4e2f-aa0d-9e2304dc14d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:31.865219 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c371a3-e5f5-4e2f-aa0d-9e2304dc14d0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:31.865510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c371a3-e5f5-4e2f-aa0d-9e2304dc14d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:31.865790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c371a3-e5f5-4e2f-aa0d-9e2304dc14d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:31.973790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c371a3-e5f5-4e2f-aa0d-9e2304dc14d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:31.975577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c371a3-e5f5-4e2f-aa0d-9e2304dc14d0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:32.056426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:32.121987 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 77/154] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:32.128062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8921ae-1acf-4abc-b11a-93033077eb62 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:32.199371 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce22354-05b3-4b49-ba0d-2cb8456b609e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:32.199718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce22354-05b3-4b49-ba0d-2cb8456b609e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:32.199838 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce22354-05b3-4b49-ba0d-2cb8456b609e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:32.201609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 78/155] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:32.212324 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a34b9-127c-4603-b120-40fcc88d17b1 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:32.212742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a34b9-127c-4603-b120-40fcc88d17b1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:32.212890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a34b9-127c-4603-b120-40fcc88d17b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:32.244760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-744a34b9-127c-4603-b120-40fcc88d17b1 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:32.246863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-744a34b9-127c-4603-b120-40fcc88d17b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:32.315998 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8921ae-1acf-4abc-b11a-93033077eb62 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:32.427000 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143d438-b9bb-4fcb-b5cf-5768b7a5dc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:32.429753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143d438-b9bb-4fcb-b5cf-5768b7a5dc8a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:32.431131 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143d438-b9bb-4fcb-b5cf-5768b7a5dc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:32.431953 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143d438-b9bb-4fcb-b5cf-5768b7a5dc8a None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:32.437795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 79/156] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:55:32.437795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5ffda-4d7c-4a8b-a50a-e0871926230b None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:32.437795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5ffda-4d7c-4a8b-a50a-e0871926230b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:32.437795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5ffda-4d7c-4a8b-a50a-e0871926230b None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:32.438362 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 80/157] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:32.440543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143d438-b9bb-4fcb-b5cf-5768b7a5dc8a None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:32.443696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143d438-b9bb-4fcb-b5cf-5768b7a5dc8a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:32.444031 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df78e6-28c3-4291-a4ad-6ab25b05f1ff None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:32.444304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df78e6-28c3-4291-a4ad-6ab25b05f1ff None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:32.444577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df78e6-28c3-4291-a4ad-6ab25b05f1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:32.448688 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 78/158] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:32 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:32.458100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143d438-b9bb-4fcb-b5cf-5768b7a5dc8a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:32.476981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-14df78e6-28c3-4291-a4ad-6ab25b05f1ff None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:32.478210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-14df78e6-28c3-4291-a4ad-6ab25b05f1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:32.576050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 81/159] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:55:32.577307 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df78e6-28c3-4291-a4ad-6ab25b05f1ff None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:32.608050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:32.613924 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:32.751450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143d438-b9bb-4fcb-b5cf-5768b7a5dc8a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:32.829035 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df78e6-28c3-4291-a4ad-6ab25b05f1ff None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:32.836391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4e04a-644b-40fa-9fc0-f23ea5ad1684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:32.837074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4e04a-644b-40fa-9fc0-f23ea5ad1684 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:32.837363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4e04a-644b-40fa-9fc0-f23ea5ad1684 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:32.837643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4e04a-644b-40fa-9fc0-f23ea5ad1684 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:32.838497 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4e04a-644b-40fa-9fc0-f23ea5ad1684 None admin] RBAC: Adding query filter params (role.id=5a989bb8c8a1465ca76622ffb504dee5, scope.project.id=e1099d961ac5441c973dfbd04812f431, user.id=e430ce5b82254dcb8def7ae26a6c6e27) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:32.838816 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4e04a-644b-40fa-9fc0-f23ea5ad1684 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:32.840368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4e04a-644b-40fa-9fc0-f23ea5ad1684 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:32.860141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 79/160] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:32 2022] GET /identity/v3/role_assignments?user.id=e430ce5b82254dcb8def7ae26a6c6e27&scope.project.id=e1099d961ac5441c973dfbd04812f431&role.id=5a989bb8c8a1465ca76622ffb504dee5 => generated 608 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:32.862045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4e04a-644b-40fa-9fc0-f23ea5ad1684 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:32.913159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca249f35-563f-4a92-a3ef-e7b19a40454c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:32.914664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca249f35-563f-4a92-a3ef-e7b19a40454c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:32.915080 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca249f35-563f-4a92-a3ef-e7b19a40454c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:32.915481 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca249f35-563f-4a92-a3ef-e7b19a40454c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:32.956973 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca249f35-563f-4a92-a3ef-e7b19a40454c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-ca249f35-563f-4a92-a3ef-e7b19a40454c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:32.981588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:32.983293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:32.984881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:32.984881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:32.984881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:32.984881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:32.984881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:32.984881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:32.984881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:32.984881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:32.984881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:32.984881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:32.984881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:32.984881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:32.984881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 82/161] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:55:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:32.999479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca249f35-563f-4a92-a3ef-e7b19a40454c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:33.114398 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:33.157742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca249f35-563f-4a92-a3ef-e7b19a40454c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:33.221479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5da425-d48a-4bb3-82ae-56994076125a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:33.222211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5da425-d48a-4bb3-82ae-56994076125a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:33.222502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5da425-d48a-4bb3-82ae-56994076125a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:33.222763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5da425-d48a-4bb3-82ae-56994076125a None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:33.223541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5da425-d48a-4bb3-82ae-56994076125a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:33.223853 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5da425-d48a-4bb3-82ae-56994076125a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:33.225295 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5da425-d48a-4bb3-82ae-56994076125a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:33.239745 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4e04a-644b-40fa-9fc0-f23ea5ad1684 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:33.240856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 83/162] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:33.251096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5da425-d48a-4bb3-82ae-56994076125a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:33.293218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:33.381320 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5da425-d48a-4bb3-82ae-56994076125a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:33.381557 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a20553-832c-4e34-b176-0227b13f93d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:33.382514 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a20553-832c-4e34-b176-0227b13f93d5 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:33.383015 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a20553-832c-4e34-b176-0227b13f93d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:33.383520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a20553-832c-4e34-b176-0227b13f93d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:33.436943 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a20553-832c-4e34-b176-0227b13f93d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:33.441843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f43382-9bfc-4726-a713-2d415220e1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:33.442732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f43382-9bfc-4726-a713-2d415220e1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:33.443021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f43382-9bfc-4726-a713-2d415220e1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:33.443298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f43382-9bfc-4726-a713-2d415220e1c9 None admin] PATH_INFO: `/v3/users/af6fa97a676b4e53bc4e27a552d09f11` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:33.445919 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f43382-9bfc-4726-a713-2d415220e1c9 None admin] RBAC: Authorizing `identity:get_user(user_id=af6fa97a676b4e53bc4e27a552d09f11)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:33.447478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f43382-9bfc-4726-a713-2d415220e1c9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:33.450758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 84/163] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:33 2022] GET /identity/v3/users/af6fa97a676b4e53bc4e27a552d09f11 => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-a4a20553-832c-4e34-b176-0227b13f93d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:33.458077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:33.458987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:33.459731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:33.459731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:33.459731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:33.459731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:33.459731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:33.459731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:33.459731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:33.459731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:33.459731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:33.459731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:33.459731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:33.459731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:33.461093 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f43382-9bfc-4726-a713-2d415220e1c9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:33.461188 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 80/164] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 599 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:33.462565 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:33.473952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a20553-832c-4e34-b176-0227b13f93d5 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:33.624809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f43382-9bfc-4726-a713-2d415220e1c9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:33.693324 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3112bf-db6a-4358-a5cd-28db9ac18173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:33.694075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3112bf-db6a-4358-a5cd-28db9ac18173 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:33.694363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3112bf-db6a-4358-a5cd-28db9ac18173 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:33.694650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3112bf-db6a-4358-a5cd-28db9ac18173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:33.756671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a20553-832c-4e34-b176-0227b13f93d5 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:33.819132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3112bf-db6a-4358-a5cd-28db9ac18173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:33.825685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3112bf-db6a-4358-a5cd-28db9ac18173 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:33.826054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97564a0f-e4e4-4d6f-ab44-49d2333959e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:33.827087 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97564a0f-e4e4-4d6f-ab44-49d2333959e3 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:33.827544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97564a0f-e4e4-4d6f-ab44-49d2333959e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:33.828103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97564a0f-e4e4-4d6f-ab44-49d2333959e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:33.829464 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97564a0f-e4e4-4d6f-ab44-49d2333959e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:33.829995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97564a0f-e4e4-4d6f-ab44-49d2333959e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:33.832805 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97564a0f-e4e4-4d6f-ab44-49d2333959e3 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:33.840407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:33.852165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 81/165] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:33.865448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97564a0f-e4e4-4d6f-ab44-49d2333959e3 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:33.961620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:34.101930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 85/166] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:34.103293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab54c5-e8e0-40f2-ab80-355661c31d1d None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:34.103593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab54c5-e8e0-40f2-ab80-355661c31d1d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:34.103858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab54c5-e8e0-40f2-ab80-355661c31d1d None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:34.105826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 86/167] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:34.115315 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab54c5-e8e0-40f2-ab80-355661c31d1d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:34.118833 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97564a0f-e4e4-4d6f-ab44-49d2333959e3 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:34.182715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b019c8-58b5-41f0-ae84-1b1fd70b226b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:34.184368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b019c8-58b5-41f0-ae84-1b1fd70b226b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:34.184594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b019c8-58b5-41f0-ae84-1b1fd70b226b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:34.184809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b019c8-58b5-41f0-ae84-1b1fd70b226b None admin] PATH_INFO: `/v3/users/e430ce5b82254dcb8def7ae26a6c6e27` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:34.186190 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b019c8-58b5-41f0-ae84-1b1fd70b226b None admin] RBAC: Authorizing `identity:get_user(user_id=e430ce5b82254dcb8def7ae26a6c6e27)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:34.187707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b019c8-58b5-41f0-ae84-1b1fd70b226b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:34.189854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 82/168] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:33 2022] GET /identity/v3/users/e430ce5b82254dcb8def7ae26a6c6e27 => generated 275 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:34.191243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629492f-6256-4f6b-95ca-f4c734f206b7 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:34.191537 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629492f-6256-4f6b-95ca-f4c734f206b7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:34.191811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629492f-6256-4f6b-95ca-f4c734f206b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:34.209537 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d629492f-6256-4f6b-95ca-f4c734f206b7 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:34.210900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d629492f-6256-4f6b-95ca-f4c734f206b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:34.313412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 83/169] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:34.314218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629492f-6256-4f6b-95ca-f4c734f206b7 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:34.360052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab54c5-e8e0-40f2-ab80-355661c31d1d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:34.365059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:34.383395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:34.464755 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acde2d-8694-4559-86a0-96da5f54f246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:34.465650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acde2d-8694-4559-86a0-96da5f54f246 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:34.465938 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acde2d-8694-4559-86a0-96da5f54f246 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:34.466339 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acde2d-8694-4559-86a0-96da5f54f246 None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:34.467691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acde2d-8694-4559-86a0-96da5f54f246 None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:34.469093 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acde2d-8694-4559-86a0-96da5f54f246 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:34.471070 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 87/170] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:34 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:34.473677 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6b66e-ac0d-4a0b-b667-d528cc7b74ca None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:34.473963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6b66e-ac0d-4a0b-b667-d528cc7b74ca None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:34.474244 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6b66e-ac0d-4a0b-b667-d528cc7b74ca None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:34.475639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 88/171] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:34.479376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6b66e-ac0d-4a0b-b667-d528cc7b74ca None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:34.480310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629492f-6256-4f6b-95ca-f4c734f206b7 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:34.531081 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2922312-e28f-4fb9-ae10-bc9b0a65a385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:34.533336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2922312-e28f-4fb9-ae10-bc9b0a65a385 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:34.535482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2922312-e28f-4fb9-ae10-bc9b0a65a385 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:34.536061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2922312-e28f-4fb9-ae10-bc9b0a65a385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:34.697133 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2922312-e28f-4fb9-ae10-bc9b0a65a385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:34.699169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2922312-e28f-4fb9-ae10-bc9b0a65a385 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:34.743858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6b66e-ac0d-4a0b-b667-d528cc7b74ca None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:34.788927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:34.799932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2d086-b824-421c-8d0e-8a61735fb4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:34.800922 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2d086-b824-421c-8d0e-8a61735fb4ea None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:34.801205 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2d086-b824-421c-8d0e-8a61735fb4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:34.801475 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f2d086-b824-421c-8d0e-8a61735fb4ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:34.831474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f2d086-b824-421c-8d0e-8a61735fb4ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-45f2d086-b824-421c-8d0e-8a61735fb4ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:34.839664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:34.840921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:34.841691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:34.841691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:34.841691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:34.841691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:34.841691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:34.841691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:34.841691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:34.841691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:34.841691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:34.841691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:34.841691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:34.841691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:34.841691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 89/172] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:55:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:34.854430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2d086-b824-421c-8d0e-8a61735fb4ea None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:34.915354 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 84/173] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:34.916542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e6b96-4fea-4eb3-a762-c39dca820afd None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:34.916833 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e6b96-4fea-4eb3-a762-c39dca820afd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:34.917111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e6b96-4fea-4eb3-a762-c39dca820afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:34.942641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:34.957716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6d0e6b96-4fea-4eb3-a762-c39dca820afd None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:34.959752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6d0e6b96-4fea-4eb3-a762-c39dca820afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:35.052636 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f2d086-b824-421c-8d0e-8a61735fb4ea None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:35.099438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203377b5-6ca5-4ccb-bad1-893f6030ecd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:35.100694 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203377b5-6ca5-4ccb-bad1-893f6030ecd5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:35.101539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203377b5-6ca5-4ccb-bad1-893f6030ecd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:35.102029 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203377b5-6ca5-4ccb-bad1-893f6030ecd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:35.103450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203377b5-6ca5-4ccb-bad1-893f6030ecd5 None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=e1099d961ac5441c973dfbd04812f431, user.id=af6fa97a676b4e53bc4e27a552d09f11) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:35.104237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203377b5-6ca5-4ccb-bad1-893f6030ecd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:35.107714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203377b5-6ca5-4ccb-bad1-893f6030ecd5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:35.117360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 90/174] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:34 2022] GET /identity/v3/role_assignments?user.id=af6fa97a676b4e53bc4e27a552d09f11&scope.project.id=e1099d961ac5441c973dfbd04812f431&role.id=000e942517f2469ba36d2e65978bc314 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:35.119322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203377b5-6ca5-4ccb-bad1-893f6030ecd5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:35.179713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 85/175] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:35.180726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e6b96-4fea-4eb3-a762-c39dca820afd None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:35.325051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:35.357019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203377b5-6ca5-4ccb-bad1-893f6030ecd5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:35.360314 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:35.378776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e6b96-4fea-4eb3-a762-c39dca820afd None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:35.411152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76670f-aa34-4056-b6c9-b88b8c4d268d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:35.412765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76670f-aa34-4056-b6c9-b88b8c4d268d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:35.413586 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76670f-aa34-4056-b6c9-b88b8c4d268d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:35.414720 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76670f-aa34-4056-b6c9-b88b8c4d268d None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:35.416092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76670f-aa34-4056-b6c9-b88b8c4d268d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:35.416927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76670f-aa34-4056-b6c9-b88b8c4d268d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:35.420960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76670f-aa34-4056-b6c9-b88b8c4d268d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:35.426297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f9628-2e4b-4617-88d5-01cff83b45bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:35.427362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f9628-2e4b-4617-88d5-01cff83b45bd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:35.427734 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f9628-2e4b-4617-88d5-01cff83b45bd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:35.428123 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f9628-2e4b-4617-88d5-01cff83b45bd None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:35.429666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f9628-2e4b-4617-88d5-01cff83b45bd None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:35.431342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f9628-2e4b-4617-88d5-01cff83b45bd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:35.434693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 86/176] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:35 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:35.439096 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f9628-2e4b-4617-88d5-01cff83b45bd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:35.443249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 91/177] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:35.446260 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356e553-ed81-4765-9a55-c3b7d868cd72 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:35.447253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356e553-ed81-4765-9a55-c3b7d868cd72 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:35.447877 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356e553-ed81-4765-9a55-c3b7d868cd72 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:35.452393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 92/178] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:35.454985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1b2eb-17ba-41b1-995c-a583063b12dc None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:35.462348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1b2eb-17ba-41b1-995c-a583063b12dc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:35.463056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1b2eb-17ba-41b1-995c-a583063b12dc None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:35.466847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 93/179] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:35 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:35.468536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1b2eb-17ba-41b1-995c-a583063b12dc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:35.628652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f9628-2e4b-4617-88d5-01cff83b45bd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:35.671769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:35.677700 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4fb219-e35c-4779-8e82-3375b0875ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:35.679001 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fb219-e35c-4779-8e82-3375b0875ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:35.679541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fb219-e35c-4779-8e82-3375b0875ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:35.679884 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4fb219-e35c-4779-8e82-3375b0875ab2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:35.692926 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4fb219-e35c-4779-8e82-3375b0875ab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-8e4fb219-e35c-4779-8e82-3375b0875ab2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:35.702273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:35.711023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:35.712062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:35.712062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:35.712062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:35.712062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:35.712062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:35.712062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:35.712062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:35.712062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:35.712062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:35.712062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:35.712062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:35.712062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:35.712062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 87/180] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:35.712062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4fb219-e35c-4779-8e82-3375b0875ab2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:35.715100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1b2eb-17ba-41b1-995c-a583063b12dc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:35.766314 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c3e56-541c-40b3-92bc-07b287ae89fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:35.767000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c3e56-541c-40b3-92bc-07b287ae89fd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:35.767286 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c3e56-541c-40b3-92bc-07b287ae89fd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:35.767564 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c3e56-541c-40b3-92bc-07b287ae89fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:35.768486 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c3e56-541c-40b3-92bc-07b287ae89fd None admin] RBAC: Adding query filter params (role.id=3d7c80cf23864f58bf21f893ba7691d6, scope.project.id=e1099d961ac5441c973dfbd04812f431, user.id=e430ce5b82254dcb8def7ae26a6c6e27) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:35.768805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c3e56-541c-40b3-92bc-07b287ae89fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:35.770393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c3e56-541c-40b3-92bc-07b287ae89fd None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:35.778937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 94/181] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:35 2022] GET /identity/v3/role_assignments?user.id=e430ce5b82254dcb8def7ae26a6c6e27&scope.project.id=e1099d961ac5441c973dfbd04812f431&role.id=3d7c80cf23864f58bf21f893ba7691d6 => generated 608 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:35.780366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274464a5-e645-438a-9c91-84a5e9f70470 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:35.780655 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274464a5-e645-438a-9c91-84a5e9f70470 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:35.780936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274464a5-e645-438a-9c91-84a5e9f70470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:35.816201 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-274464a5-e645-438a-9c91-84a5e9f70470 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:35.818497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-274464a5-e645-438a-9c91-84a5e9f70470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:35.849480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:35.857775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4fb219-e35c-4779-8e82-3375b0875ab2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:35.911631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a2d07-4da3-4f59-b13e-d7b5fcfb51b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:35.913059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a2d07-4da3-4f59-b13e-d7b5fcfb51b0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:35.913389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a2d07-4da3-4f59-b13e-d7b5fcfb51b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:35.913680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10a2d07-4da3-4f59-b13e-d7b5fcfb51b0 None admin] PATH_INFO: `/v3/users/2eda438a5a214b5ebec919f278e5489e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:35.927376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 95/182] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:35.932897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9725478-7530-4d23-ac65-379a025c85c7 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:35.932897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9725478-7530-4d23-ac65-379a025c85c7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:35.932897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9725478-7530-4d23-ac65-379a025c85c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:35.945773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a2d07-4da3-4f59-b13e-d7b5fcfb51b0 None admin] RBAC: Authorizing `identity:get_user(user_id=2eda438a5a214b5ebec919f278e5489e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:35.948315 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10a2d07-4da3-4f59-b13e-d7b5fcfb51b0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:35.952164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 88/183] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:35 2022] GET /identity/v3/users/2eda438a5a214b5ebec919f278e5489e => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:35.953869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a2d07-4da3-4f59-b13e-d7b5fcfb51b0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:35.965022 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e9725478-7530-4d23-ac65-379a025c85c7 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:35.966840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e9725478-7530-4d23-ac65-379a025c85c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:36.038927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:36.064616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 96/184] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:36.066982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881e6b4-656b-4595-a950-8d983f14cc68 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:36.067514 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881e6b4-656b-4595-a950-8d983f14cc68 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:36.068050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881e6b4-656b-4595-a950-8d983f14cc68 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:36.070569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 97/185] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:36.074862 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881e6b4-656b-4595-a950-8d983f14cc68 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:36.107448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10a2d07-4da3-4f59-b13e-d7b5fcfb51b0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:36.146897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae9cd0-20a0-440c-bfcb-1579c921f9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:36.147562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae9cd0-20a0-440c-bfcb-1579c921f9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:36.147843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae9cd0-20a0-440c-bfcb-1579c921f9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:36.148147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae9cd0-20a0-440c-bfcb-1579c921f9c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:36.148906 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae9cd0-20a0-440c-bfcb-1579c921f9c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:36.149206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae9cd0-20a0-440c-bfcb-1579c921f9c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:36.150596 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae9cd0-20a0-440c-bfcb-1579c921f9c3 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:36.159772 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 89/186] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:36.164848 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae9cd0-20a0-440c-bfcb-1579c921f9c3 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:36.185149 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:36.213173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881e6b4-656b-4595-a950-8d983f14cc68 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:36.258298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7844fa1-d988-4348-9f84-cbe41d494858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:36.259125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7844fa1-d988-4348-9f84-cbe41d494858 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:36.259394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7844fa1-d988-4348-9f84-cbe41d494858 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:36.259664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7844fa1-d988-4348-9f84-cbe41d494858 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:36.272337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7844fa1-d988-4348-9f84-cbe41d494858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-a7844fa1-d988-4348-9f84-cbe41d494858 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:36.281161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:36.283045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:36.284053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:36.284053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:36.284053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:36.284053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:36.284053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:36.284053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:36.284053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:36.284053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:36.284053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:36.284053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:36.284053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:55:36.284053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:36.284652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae9cd0-20a0-440c-bfcb-1579c921f9c3 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:36.284723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 98/187] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:55:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:36.284723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb0368-0702-47b4-b92e-910587014e53 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:36.284723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb0368-0702-47b4-b92e-910587014e53 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:36.284723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb0368-0702-47b4-b92e-910587014e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:36.302339 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:36.302608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c6fb0368-0702-47b4-b92e-910587014e53 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:36.304151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c6fb0368-0702-47b4-b92e-910587014e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:36.326704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05af0c-2814-4eee-8b18-6ca76159ea6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:36.327367 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05af0c-2814-4eee-8b18-6ca76159ea6e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:36.327652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05af0c-2814-4eee-8b18-6ca76159ea6e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:36.327951 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05af0c-2814-4eee-8b18-6ca76159ea6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:36.394948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 99/188] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:36.395819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb0368-0702-47b4-b92e-910587014e53 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:36.432273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db05af0c-2814-4eee-8b18-6ca76159ea6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:36.434396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db05af0c-2814-4eee-8b18-6ca76159ea6e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:36.484761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:36.520702 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb0368-0702-47b4-b92e-910587014e53 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:36.552905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 90/189] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:36.554212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05af0c-2814-4eee-8b18-6ca76159ea6e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:36.564805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea37c8-e2e3-46fc-8409-f2560d9d49e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:36.565771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea37c8-e2e3-46fc-8409-f2560d9d49e2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:36.566065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea37c8-e2e3-46fc-8409-f2560d9d49e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:36.566363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea37c8-e2e3-46fc-8409-f2560d9d49e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:36.567769 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea37c8-e2e3-46fc-8409-f2560d9d49e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:36.569286 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea37c8-e2e3-46fc-8409-f2560d9d49e2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:36.570950 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 100/190] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:55:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:36.571731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea37c8-e2e3-46fc-8409-f2560d9d49e2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:36.575450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:36.674542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05af0c-2814-4eee-8b18-6ca76159ea6e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:36.691347 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea37c8-e2e3-46fc-8409-f2560d9d49e2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:36.727844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2bac3-ab04-45c5-b0e3-4ef2857887f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:36.729309 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2bac3-ab04-45c5-b0e3-4ef2857887f2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:36.729780 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2bac3-ab04-45c5-b0e3-4ef2857887f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:36.730263 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2bac3-ab04-45c5-b0e3-4ef2857887f2 None admin] PATH_INFO: `/v3/users/e430ce5b82254dcb8def7ae26a6c6e27` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:36.732147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2bac3-ab04-45c5-b0e3-4ef2857887f2 None admin] RBAC: Authorizing `identity:get_user(user_id=e430ce5b82254dcb8def7ae26a6c6e27)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:36.734169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2bac3-ab04-45c5-b0e3-4ef2857887f2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:36.737168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 91/191] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:36 2022] GET /identity/v3/users/e430ce5b82254dcb8def7ae26a6c6e27 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:36.738323 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2bac3-ab04-45c5-b0e3-4ef2857887f2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:36.744238 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473a659-a954-4703-bc66-f4a75937c6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:36.744934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473a659-a954-4703-bc66-f4a75937c6ce None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:36.745305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473a659-a954-4703-bc66-f4a75937c6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:36.745766 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473a659-a954-4703-bc66-f4a75937c6ce None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:36.746773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473a659-a954-4703-bc66-f4a75937c6ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:36.747083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473a659-a954-4703-bc66-f4a75937c6ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:36.748538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473a659-a954-4703-bc66-f4a75937c6ce None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:36.757817 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 101/192] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:55:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:36.758649 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473a659-a954-4703-bc66-f4a75937c6ce None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:36.762238 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:36.837876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:36.860830 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2bac3-ab04-45c5-b0e3-4ef2857887f2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:36.869955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473a659-a954-4703-bc66-f4a75937c6ce None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:36.907803 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a8eb3-5ce5-486a-bddf-81022006e55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:36.908273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3818a88-6699-4f8e-8260-e115f972f0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:36.908651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a8eb3-5ce5-486a-bddf-81022006e55c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:36.908938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a8eb3-5ce5-486a-bddf-81022006e55c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:36.909148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3818a88-6699-4f8e-8260-e115f972f0ae None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:36.909281 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a8eb3-5ce5-486a-bddf-81022006e55c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:36.909427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3818a88-6699-4f8e-8260-e115f972f0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:36.909796 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3818a88-6699-4f8e-8260-e115f972f0ae None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:36.910915 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3818a88-6699-4f8e-8260-e115f972f0ae None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:36.912445 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3818a88-6699-4f8e-8260-e115f972f0ae None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:36.914219 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 102/193] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:36 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:36.915294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3818a88-6699-4f8e-8260-e115f972f0ae None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:36.927817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a8eb3-5ce5-486a-bddf-81022006e55c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-6e4a8eb3-5ce5-486a-bddf-81022006e55c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:36.937907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:36.946102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:36.947608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:36.947608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:36.947608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:36.947608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:36.947608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:36.947608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:36.947608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:36.947608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:36.947608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:36.947608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:36.947608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:36.947608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:36.947608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 92/194] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:36.947608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a8eb3-5ce5-486a-bddf-81022006e55c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:37.002799 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:37.041153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3818a88-6699-4f8e-8260-e115f972f0ae None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:37.066445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a8eb3-5ce5-486a-bddf-81022006e55c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:37.084470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d906c-2341-4ecd-8971-5dc45dab31e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:37.085196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d906c-2341-4ecd-8971-5dc45dab31e5 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:37.085555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d906c-2341-4ecd-8971-5dc45dab31e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:37.085936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d906c-2341-4ecd-8971-5dc45dab31e5 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:37.086944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d906c-2341-4ecd-8971-5dc45dab31e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:37.088626 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d906c-2341-4ecd-8971-5dc45dab31e5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:37.110639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c24b6-a922-44ac-9c68-737d8dd5a30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:37.111317 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c24b6-a922-44ac-9c68-737d8dd5a30b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:37.111603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c24b6-a922-44ac-9c68-737d8dd5a30b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:37.111881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c24b6-a922-44ac-9c68-737d8dd5a30b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:37.113530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 103/195] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:55:36 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:55:37.114440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d906c-2341-4ecd-8971-5dc45dab31e5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:37.127238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:37.219478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c24b6-a922-44ac-9c68-737d8dd5a30b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:37.222316 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c24b6-a922-44ac-9c68-737d8dd5a30b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:37.285234 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d906c-2341-4ecd-8971-5dc45dab31e5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:37.325301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:37.339948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305d4ac-c70c-4a6b-ba42-2bd9d47569db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:37.340446 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305d4ac-c70c-4a6b-ba42-2bd9d47569db None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:37.340731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305d4ac-c70c-4a6b-ba42-2bd9d47569db None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:37.341011 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305d4ac-c70c-4a6b-ba42-2bd9d47569db None admin] PATH_INFO: `/v3/users/2eda438a5a214b5ebec919f278e5489e` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:37.342409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305d4ac-c70c-4a6b-ba42-2bd9d47569db None admin] RBAC: Authorizing `identity:get_user(user_id=2eda438a5a214b5ebec919f278e5489e)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:37.344010 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305d4ac-c70c-4a6b-ba42-2bd9d47569db None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:37.345934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 104/196] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:37 2022] GET /identity/v3/users/2eda438a5a214b5ebec919f278e5489e => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:37.346558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305d4ac-c70c-4a6b-ba42-2bd9d47569db None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:37.388134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 93/197] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:37.390318 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c24b6-a922-44ac-9c68-737d8dd5a30b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:37.467715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:37.498517 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305d4ac-c70c-4a6b-ba42-2bd9d47569db None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:37.545903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a843de-69f4-4a81-85ac-63f052824e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:37.547129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a843de-69f4-4a81-85ac-63f052824e5a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:37.547428 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a843de-69f4-4a81-85ac-63f052824e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:37.547716 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a843de-69f4-4a81-85ac-63f052824e5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:37.548927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a843de-69f4-4a81-85ac-63f052824e5a None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=551f963b02f4411395171476c9cc2d68, user.id=2eda438a5a214b5ebec919f278e5489e) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:37.549251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a843de-69f4-4a81-85ac-63f052824e5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:37.551077 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a843de-69f4-4a81-85ac-63f052824e5a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:37.559569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c24b6-a922-44ac-9c68-737d8dd5a30b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:37.559758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 105/198] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:37 2022] GET /identity/v3/role_assignments?user.id=2eda438a5a214b5ebec919f278e5489e&scope.project.id=551f963b02f4411395171476c9cc2d68&role.id=000e942517f2469ba36d2e65978bc314 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:37.560726 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a843de-69f4-4a81-85ac-63f052824e5a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:37.619812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938def1-d96f-45ea-83d8-4d485bc3cd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:37.621633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938def1-d96f-45ea-83d8-4d485bc3cd2d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:37.621839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938def1-d96f-45ea-83d8-4d485bc3cd2d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:37.622343 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938def1-d96f-45ea-83d8-4d485bc3cd2d None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:37.623902 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938def1-d96f-45ea-83d8-4d485bc3cd2d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:37.624389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938def1-d96f-45ea-83d8-4d485bc3cd2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:37.626999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938def1-d96f-45ea-83d8-4d485bc3cd2d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:37.642919 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:37.647597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 94/199] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:37.650115 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938def1-d96f-45ea-83d8-4d485bc3cd2d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:37.695508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a843de-69f4-4a81-85ac-63f052824e5a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:37.790075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec2b66-c16f-432a-b5d9-e324d38c3a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:37.791104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec2b66-c16f-432a-b5d9-e324d38c3a28 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:37.791598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec2b66-c16f-432a-b5d9-e324d38c3a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:37.792164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec2b66-c16f-432a-b5d9-e324d38c3a28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:37.875501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938def1-d96f-45ea-83d8-4d485bc3cd2d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:37.878679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:37.941213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec2b66-c16f-432a-b5d9-e324d38c3a28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:37.942717 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec2b66-c16f-432a-b5d9-e324d38c3a28 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:37.947512 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ac603-b6e2-43e9-beab-dfd6fa8aaada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:37.949624 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ac603-b6e2-43e9-beab-dfd6fa8aaada None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:37.950131 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ac603-b6e2-43e9-beab-dfd6fa8aaada None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:37.950637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ac603-b6e2-43e9-beab-dfd6fa8aaada None admin] PATH_INFO: `/v3/projects/a66f42db6df147daa57765d7d45ade8c` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:37.952940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ac603-b6e2-43e9-beab-dfd6fa8aaada None admin] RBAC: Authorizing `identity:get_project(project_id=a66f42db6df147daa57765d7d45ade8c)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:37.955711 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ac603-b6e2-43e9-beab-dfd6fa8aaada None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:37.959018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 95/200] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:37 2022] GET /identity/v3/projects/a66f42db6df147daa57765d7d45ade8c => generated 312 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:37.960626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ac603-b6e2-43e9-beab-dfd6fa8aaada None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:38.097924 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:38.156479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ac603-b6e2-43e9-beab-dfd6fa8aaada None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:38.185174 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 106/201] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:38.190723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec2b66-c16f-432a-b5d9-e324d38c3a28 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:38.248538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec07bf-662c-477b-888d-179321f6284f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:38.250112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec07bf-662c-477b-888d-179321f6284f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:38.250632 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec07bf-662c-477b-888d-179321f6284f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:38.251131 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec07bf-662c-477b-888d-179321f6284f None admin] PATH_INFO: `/v3/users/7ba631c71055479f9cadb3aeaf8d8d68` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:38.253759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec07bf-662c-477b-888d-179321f6284f None admin] RBAC: Authorizing `identity:get_user(user_id=7ba631c71055479f9cadb3aeaf8d8d68)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:38.256410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec07bf-662c-477b-888d-179321f6284f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:38.259603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 96/202] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:37 2022] GET /identity/v3/users/7ba631c71055479f9cadb3aeaf8d8d68 => generated 289 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:38.269909 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec07bf-662c-477b-888d-179321f6284f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:38.354039 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:38.425904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec07bf-662c-477b-888d-179321f6284f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:38.467153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a3017-b88e-423b-ba0c-e6956a6d4dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:38.468963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a3017-b88e-423b-ba0c-e6956a6d4dcd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:38.469514 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a3017-b88e-423b-ba0c-e6956a6d4dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:38.469992 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a3017-b88e-423b-ba0c-e6956a6d4dcd None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:38.471538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a3017-b88e-423b-ba0c-e6956a6d4dcd None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:38.473288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a3017-b88e-423b-ba0c-e6956a6d4dcd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:38.475736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 97/203] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:38 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:38.477528 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a3017-b88e-423b-ba0c-e6956a6d4dcd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:38.542272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec2b66-c16f-432a-b5d9-e324d38c3a28 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:38.597627 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a3017-b88e-423b-ba0c-e6956a6d4dcd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:38.621584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:38.639809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a740afa-e173-45f2-910a-5d49c7dce5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:38.640895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a740afa-e173-45f2-910a-5d49c7dce5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:38.641459 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a740afa-e173-45f2-910a-5d49c7dce5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:38.641941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a740afa-e173-45f2-910a-5d49c7dce5d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:38.661327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bc4a5-f755-4786-9abd-7cce6a5d7bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:38.662021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bc4a5-f755-4786-9abd-7cce6a5d7bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:38.662306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bc4a5-f755-4786-9abd-7cce6a5d7bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:38.662582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bc4a5-f755-4786-9abd-7cce6a5d7bd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:38.663451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bc4a5-f755-4786-9abd-7cce6a5d7bd5 None admin] RBAC: Adding query filter params (role.id=3d7c80cf23864f58bf21f893ba7691d6, scope.project.id=a66f42db6df147daa57765d7d45ade8c, user.id=e430ce5b82254dcb8def7ae26a6c6e27) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:38.663769 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bc4a5-f755-4786-9abd-7cce6a5d7bd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:38.665742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bc4a5-f755-4786-9abd-7cce6a5d7bd5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:38.680182 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 107/204] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:38 2022] GET /identity/v3/role_assignments?user.id=e430ce5b82254dcb8def7ae26a6c6e27&scope.project.id=a66f42db6df147daa57765d7d45ade8c&role.id=3d7c80cf23864f58bf21f893ba7691d6 => generated 608 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:38.682004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bc4a5-f755-4786-9abd-7cce6a5d7bd5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:38.730955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:38.731768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a740afa-e173-45f2-910a-5d49c7dce5d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:38.733706 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a740afa-e173-45f2-910a-5d49c7dce5d8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:38.867240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 98/205] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:38.883699 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bc4a5-f755-4786-9abd-7cce6a5d7bd5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:38.888967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a740afa-e173-45f2-910a-5d49c7dce5d8 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:38.898491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:38.962114 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc20bcd-9eed-4d70-aad2-d367aca6b8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:38.963167 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc20bcd-9eed-4d70-aad2-d367aca6b8aa None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:38.964096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc20bcd-9eed-4d70-aad2-d367aca6b8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:38.964654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc20bcd-9eed-4d70-aad2-d367aca6b8aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:38.965775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc20bcd-9eed-4d70-aad2-d367aca6b8aa None admin] RBAC: Adding query filter params (role.id=5a989bb8c8a1465ca76622ffb504dee5, scope.project.id=551f963b02f4411395171476c9cc2d68, user.id=2eda438a5a214b5ebec919f278e5489e) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:38.966308 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc20bcd-9eed-4d70-aad2-d367aca6b8aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:38.968135 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc20bcd-9eed-4d70-aad2-d367aca6b8aa None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:38.975835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 108/206] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:38 2022] GET /identity/v3/role_assignments?user.id=2eda438a5a214b5ebec919f278e5489e&scope.project.id=551f963b02f4411395171476c9cc2d68&role.id=5a989bb8c8a1465ca76622ffb504dee5 => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:39.093426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a740afa-e173-45f2-910a-5d49c7dce5d8 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:39.150342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035d572-f5f8-4fba-b7a7-6f13a0c48ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:39.152196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035d572-f5f8-4fba-b7a7-6f13a0c48ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:39.152779 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035d572-f5f8-4fba-b7a7-6f13a0c48ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:39.152987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035d572-f5f8-4fba-b7a7-6f13a0c48ea9 None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:39.157208 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035d572-f5f8-4fba-b7a7-6f13a0c48ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:39.166768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035d572-f5f8-4fba-b7a7-6f13a0c48ea9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:39.166768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 99/207] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:38 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:39.169901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc20bcd-9eed-4d70-aad2-d367aca6b8aa None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:39.324052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:39.347384 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc20bcd-9eed-4d70-aad2-d367aca6b8aa None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:39.410667 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d4138-8cee-4a2b-a6a0-902700b42439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:39.411270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d4138-8cee-4a2b-a6a0-902700b42439 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:39.411557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d4138-8cee-4a2b-a6a0-902700b42439 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:39.411836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d4138-8cee-4a2b-a6a0-902700b42439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:39.413094 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d4138-8cee-4a2b-a6a0-902700b42439 None admin] RBAC: Adding query filter params (role.id=0a6e69b6ee9e43659f71a5ba3260f5c9, scope.project.id=e1099d961ac5441c973dfbd04812f431, user.id=7ba631c71055479f9cadb3aeaf8d8d68) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:39.413440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d4138-8cee-4a2b-a6a0-902700b42439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:39.415334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d4138-8cee-4a2b-a6a0-902700b42439 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:39.429609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 109/208] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:39 2022] GET /identity/v3/role_assignments?user.id=7ba631c71055479f9cadb3aeaf8d8d68&scope.project.id=e1099d961ac5441c973dfbd04812f431&role.id=0a6e69b6ee9e43659f71a5ba3260f5c9 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:40.218021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df13e3-37b9-49de-a3cf-704cda889b3b None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:40.218574 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df13e3-37b9-49de-a3cf-704cda889b3b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:40.218574 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df13e3-37b9-49de-a3cf-704cda889b3b None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:40.219973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 100/209] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:40.225949 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3764158-143c-4726-b1f4-cb3c3b48aa0d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:40.226445 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3764158-143c-4726-b1f4-cb3c3b48aa0d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:40.226536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3764158-143c-4726-b1f4-cb3c3b48aa0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:40.244747 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a3764158-143c-4726-b1f4-cb3c3b48aa0d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:40.246037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a3764158-143c-4726-b1f4-cb3c3b48aa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:40.334268 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 110/210] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:40.357890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df13e3-37b9-49de-a3cf-704cda889b3b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:40.373705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab322b0-d246-4b3d-9ab3-b55dac72a5f4 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:40.374118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab322b0-d246-4b3d-9ab3-b55dac72a5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:40.374409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab322b0-d246-4b3d-9ab3-b55dac72a5f4 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:40.375853 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 111/211] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:55:40.382203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f32461-8079-44ea-b75d-7dbc6d214d0d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:40.382495 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f32461-8079-44ea-b75d-7dbc6d214d0d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:40.382783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f32461-8079-44ea-b75d-7dbc6d214d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:40.408365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-01f32461-8079-44ea-b75d-7dbc6d214d0d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:40.409606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-01f32461-8079-44ea-b75d-7dbc6d214d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:40.453112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:40.490232 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df13e3-37b9-49de-a3cf-704cda889b3b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:40.502920 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 112/212] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:40.505095 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1613e4-bb60-47b4-b3d2-99abd2add870 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:40.505814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1613e4-bb60-47b4-b3d2-99abd2add870 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:40.506217 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1613e4-bb60-47b4-b3d2-99abd2add870 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:40.507957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 113/213] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:40.514279 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b0c7a-8fc3-4532-9954-24f57031bd70 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:40.514572 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b0c7a-8fc3-4532-9954-24f57031bd70 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:40.514854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b0c7a-8fc3-4532-9954-24f57031bd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:40.532389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6a1b0c7a-8fc3-4532-9954-24f57031bd70 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:40.533826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6a1b0c7a-8fc3-4532-9954-24f57031bd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:40.537206 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:40.544342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d07db-f36b-453f-81ba-0256d3458025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:40.546221 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d07db-f36b-453f-81ba-0256d3458025 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:40.546731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d07db-f36b-453f-81ba-0256d3458025 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:40.547227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d07db-f36b-453f-81ba-0256d3458025 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:40.566130 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d07db-f36b-453f-81ba-0256d3458025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-340d07db-f36b-453f-81ba-0256d3458025 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:40.581666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:40.582735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:40.583872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:40.583872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:40.583872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:40.583872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:40.583872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:40.583872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:40.583872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:40.583872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:40.583872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:40.583872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:40.583872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:40.583872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:40.590413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 101/214] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:55:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:40.590413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d07db-f36b-453f-81ba-0256d3458025 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:40.640739 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 114/215] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:40.642324 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b0c7a-8fc3-4532-9954-24f57031bd70 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:40.746490 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d07db-f36b-453f-81ba-0256d3458025 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:40.788431 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98d4b7-4574-4a67-bfed-ae4a63d8fb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:40.789689 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98d4b7-4574-4a67-bfed-ae4a63d8fb94 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:40.790165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98d4b7-4574-4a67-bfed-ae4a63d8fb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:40.790616 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98d4b7-4574-4a67-bfed-ae4a63d8fb94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:40.793532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:40.807530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1b0c7a-8fc3-4532-9954-24f57031bd70 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:40.809290 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98d4b7-4574-4a67-bfed-ae4a63d8fb94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-0e98d4b7-4574-4a67-bfed-ae4a63d8fb94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:40.819144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:40.821060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:40.822719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:40.822719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:40.822719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:40.822719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:40.822719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:40.822719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:40.822719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:40.822719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:40.822719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:40.822719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:40.822719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:40.822719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:40.822719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 102/216] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:40.822719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98d4b7-4574-4a67-bfed-ae4a63d8fb94 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:40.857686 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e23349-d685-4cd1-8c52-5fa97f33d712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:40.858370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e23349-d685-4cd1-8c52-5fa97f33d712 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:40.858683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e23349-d685-4cd1-8c52-5fa97f33d712 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:40.858927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e23349-d685-4cd1-8c52-5fa97f33d712 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:40.859687 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e23349-d685-4cd1-8c52-5fa97f33d712 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:40.860024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e23349-d685-4cd1-8c52-5fa97f33d712 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:40.861460 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e23349-d685-4cd1-8c52-5fa97f33d712 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:40.871820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 115/217] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:40.872582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e23349-d685-4cd1-8c52-5fa97f33d712 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:40.875806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:40.971994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98d4b7-4574-4a67-bfed-ae4a63d8fb94 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:41.002964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e23349-d685-4cd1-8c52-5fa97f33d712 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:41.012404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc635c-6b24-487d-b2e6-082f77a19856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:41.013207 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc635c-6b24-487d-b2e6-082f77a19856 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:41.013496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc635c-6b24-487d-b2e6-082f77a19856 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:41.013776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc635c-6b24-487d-b2e6-082f77a19856 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:41.014982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc635c-6b24-487d-b2e6-082f77a19856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:41.016478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc635c-6b24-487d-b2e6-082f77a19856 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:41.018310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 103/218] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:55:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:41.018968 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc635c-6b24-487d-b2e6-082f77a19856 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:41.050736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc47eb-962e-4624-894f-15514640b8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:41.051895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc47eb-962e-4624-894f-15514640b8ca None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:41.052435 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc47eb-962e-4624-894f-15514640b8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:41.052930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc47eb-962e-4624-894f-15514640b8ca None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:41.053734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc47eb-962e-4624-894f-15514640b8ca None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:41.054050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc47eb-962e-4624-894f-15514640b8ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:41.055457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc47eb-962e-4624-894f-15514640b8ca None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:41.071038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 116/219] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:41.072310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc47eb-962e-4624-894f-15514640b8ca None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:41.121531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:41.173200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:41.185415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc635c-6b24-487d-b2e6-082f77a19856 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:41.236500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc47eb-962e-4624-894f-15514640b8ca None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:41.246142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7dff2-8648-45f4-9730-4d39a0e4fbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:41.247628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7dff2-8648-45f4-9730-4d39a0e4fbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:41.248116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7dff2-8648-45f4-9730-4d39a0e4fbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:41.248575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa7dff2-8648-45f4-9730-4d39a0e4fbf7 None admin] PATH_INFO: `/v3/users/2eda438a5a214b5ebec919f278e5489e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:41.251199 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa7dff2-8648-45f4-9730-4d39a0e4fbf7 None admin] RBAC: Authorizing `identity:get_user(user_id=2eda438a5a214b5ebec919f278e5489e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:41.254384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa7dff2-8648-45f4-9730-4d39a0e4fbf7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:41.265881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 104/220] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:41 2022] GET /identity/v3/users/2eda438a5a214b5ebec919f278e5489e => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:41.265881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7dff2-8648-45f4-9730-4d39a0e4fbf7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:41.293416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3867c-3e05-4807-8762-f06ffa40ba63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:41.294506 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3867c-3e05-4807-8762-f06ffa40ba63 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:41.294989 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3867c-3e05-4807-8762-f06ffa40ba63 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:41.295462 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3867c-3e05-4807-8762-f06ffa40ba63 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:41.296725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3867c-3e05-4807-8762-f06ffa40ba63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:41.299237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3867c-3e05-4807-8762-f06ffa40ba63 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:41.330097 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 117/221] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:55:41 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:55:41.331563 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3867c-3e05-4807-8762-f06ffa40ba63 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:41.482536 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa7dff2-8648-45f4-9730-4d39a0e4fbf7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:41.523008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3867c-3e05-4807-8762-f06ffa40ba63 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:41.530353 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:41.544313 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7943a39-f56e-4bc8-abc6-50dff97ec839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:41.545881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7943a39-f56e-4bc8-abc6-50dff97ec839 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:41.546223 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7943a39-f56e-4bc8-abc6-50dff97ec839 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:41.547305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7943a39-f56e-4bc8-abc6-50dff97ec839 None admin] PATH_INFO: `/v3/users/860b4fd63ee04b2fa8fc1a0c33b66ecc` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:41.559327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:41.566591 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7943a39-f56e-4bc8-abc6-50dff97ec839 None admin] RBAC: Authorizing `identity:get_user(user_id=860b4fd63ee04b2fa8fc1a0c33b66ecc)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:41.568602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7943a39-f56e-4bc8-abc6-50dff97ec839 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:41.570692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 105/222] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:41 2022] GET /identity/v3/users/860b4fd63ee04b2fa8fc1a0c33b66ecc => generated 297 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:41.577263 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7943a39-f56e-4bc8-abc6-50dff97ec839 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:41.577998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ea7bf-6b6a-4067-869a-c8a40e290498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:41.579398 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ea7bf-6b6a-4067-869a-c8a40e290498 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:41.579931 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ea7bf-6b6a-4067-869a-c8a40e290498 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:41.581063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ea7bf-6b6a-4067-869a-c8a40e290498 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:41.583743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ea7bf-6b6a-4067-869a-c8a40e290498 None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:41.586107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ea7bf-6b6a-4067-869a-c8a40e290498 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:41.596491 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 118/223] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:41 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:41.598903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ea7bf-6b6a-4067-869a-c8a40e290498 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:41.771897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7943a39-f56e-4bc8-abc6-50dff97ec839 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:41.780980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ea7bf-6b6a-4067-869a-c8a40e290498 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:41.841337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03ccde-d23a-4c5d-8a86-5f218217101b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:41.842027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03ccde-d23a-4c5d-8a86-5f218217101b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:41.842314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03ccde-d23a-4c5d-8a86-5f218217101b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:41.842597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03ccde-d23a-4c5d-8a86-5f218217101b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:41.844889 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50288d1-7136-447f-8d75-d5faf3d5af5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:41.846486 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50288d1-7136-447f-8d75-d5faf3d5af5f None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:41.846834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50288d1-7136-447f-8d75-d5faf3d5af5f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:41.847280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50288d1-7136-447f-8d75-d5faf3d5af5f None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68/users/2eda438a5a214b5ebec919f278e5489e/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:41.850968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50288d1-7136-447f-8d75-d5faf3d5af5f None admin] RBAC: Authorizing `identity:create_grant(project_id=551f963b02f4411395171476c9cc2d68, user_id=2eda438a5a214b5ebec919f278e5489e, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:41.853426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50288d1-7136-447f-8d75-d5faf3d5af5f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:41.878196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 119/224] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:55:41 2022] PUT /identity/v3/projects/551f963b02f4411395171476c9cc2d68/users/2eda438a5a214b5ebec919f278e5489e/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:55:41.880728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d6a6d-fdf4-4074-8a5e-0c6553e5f025 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:41.881114 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d6a6d-fdf4-4074-8a5e-0c6553e5f025 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:41.881408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d6a6d-fdf4-4074-8a5e-0c6553e5f025 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:41.884108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 120/225] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:41.890352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3384bed-a890-4eef-9cbc-bd86ca242ac8 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:41.890689 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3384bed-a890-4eef-9cbc-bd86ca242ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:41.890951 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3384bed-a890-4eef-9cbc-bd86ca242ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:41.929022 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b3384bed-a890-4eef-9cbc-bd86ca242ac8 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:41.930327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b3384bed-a890-4eef-9cbc-bd86ca242ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:41.950860 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:41.984800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03ccde-d23a-4c5d-8a86-5f218217101b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:41.986429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03ccde-d23a-4c5d-8a86-5f218217101b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:42.043072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:42.079441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 121/226] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:55:42.104101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3384bed-a890-4eef-9cbc-bd86ca242ac8 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:42.178800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 106/227] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:42.184769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03ccde-d23a-4c5d-8a86-5f218217101b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:42.281507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3384bed-a890-4eef-9cbc-bd86ca242ac8 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:42.339501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f359f-3c75-4a8f-8c43-b6dee0139b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:42.340310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f359f-3c75-4a8f-8c43-b6dee0139b2e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:42.340594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f359f-3c75-4a8f-8c43-b6dee0139b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:42.340877 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f359f-3c75-4a8f-8c43-b6dee0139b2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:42.354554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f359f-3c75-4a8f-8c43-b6dee0139b2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-597f359f-3c75-4a8f-8c43-b6dee0139b2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:42.365076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:42.366007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:42.366774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:42.366774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:42.366774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:42.366774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:42.366774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:42.366774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:42.366774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:42.366774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:42.366774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:42.366774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:42.366774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:55:42.366774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:42.366774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 122/228] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:55:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:42.373279 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f359f-3c75-4a8f-8c43-b6dee0139b2e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:42.385271 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:42.421381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03ccde-d23a-4c5d-8a86-5f218217101b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:42.457650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:42.523867 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a75982-07dd-44fa-865a-3c97f8789545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:42.525397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a75982-07dd-44fa-865a-3c97f8789545 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:42.525903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a75982-07dd-44fa-865a-3c97f8789545 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:42.526415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a75982-07dd-44fa-865a-3c97f8789545 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:42.532911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a75982-07dd-44fa-865a-3c97f8789545 None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:42.534440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a75982-07dd-44fa-865a-3c97f8789545 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:42.536865 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 107/229] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:42 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:42.543905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a75982-07dd-44fa-865a-3c97f8789545 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:42.594543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f359f-3c75-4a8f-8c43-b6dee0139b2e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:42.663252 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b90f03-79aa-4f45-87f4-f50efe95ece8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:42.663974 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b90f03-79aa-4f45-87f4-f50efe95ece8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:42.664522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b90f03-79aa-4f45-87f4-f50efe95ece8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:42.664804 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b90f03-79aa-4f45-87f4-f50efe95ece8 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:42.665596 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b90f03-79aa-4f45-87f4-f50efe95ece8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:42.665906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b90f03-79aa-4f45-87f4-f50efe95ece8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:42.667444 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b90f03-79aa-4f45-87f4-f50efe95ece8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:42.688970 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 123/230] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:55:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:42.697250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b90f03-79aa-4f45-87f4-f50efe95ece8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:42.729731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a75982-07dd-44fa-865a-3c97f8789545 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:42.779091 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485683c-1ac2-46f8-84e6-7354e4d3518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:42.780933 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485683c-1ac2-46f8-84e6-7354e4d3518e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:42.781613 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485683c-1ac2-46f8-84e6-7354e4d3518e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:42.783402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485683c-1ac2-46f8-84e6-7354e4d3518e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:42.789061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485683c-1ac2-46f8-84e6-7354e4d3518e None admin] RBAC: Adding query filter params (role.id=3d7c80cf23864f58bf21f893ba7691d6, scope.project.id=551f963b02f4411395171476c9cc2d68, user.id=860b4fd63ee04b2fa8fc1a0c33b66ecc) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:42.789642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485683c-1ac2-46f8-84e6-7354e4d3518e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:42.791175 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485683c-1ac2-46f8-84e6-7354e4d3518e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:42.803177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 108/231] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:42 2022] GET /identity/v3/role_assignments?user.id=860b4fd63ee04b2fa8fc1a0c33b66ecc&scope.project.id=551f963b02f4411395171476c9cc2d68&role.id=3d7c80cf23864f58bf21f893ba7691d6 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:42.922016 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:42.930539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b90f03-79aa-4f45-87f4-f50efe95ece8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:42.987576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80a162-2891-46c0-94f5-ad3d13933087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:42.988976 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80a162-2891-46c0-94f5-ad3d13933087 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:42.989504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80a162-2891-46c0-94f5-ad3d13933087 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:42.989962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80a162-2891-46c0-94f5-ad3d13933087 None admin] PATH_INFO: `/v3/users/2eda438a5a214b5ebec919f278e5489e` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:42.991845 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80a162-2891-46c0-94f5-ad3d13933087 None admin] RBAC: Authorizing `identity:get_user(user_id=2eda438a5a214b5ebec919f278e5489e)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:42.994129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80a162-2891-46c0-94f5-ad3d13933087 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:42.997460 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 124/232] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:42 2022] GET /identity/v3/users/2eda438a5a214b5ebec919f278e5489e => generated 283 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:43.003356 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485683c-1ac2-46f8-84e6-7354e4d3518e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:43.046335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:43.179685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485683c-1ac2-46f8-84e6-7354e4d3518e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:43.242247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75aa72-0a82-48a8-9f02-17cb4e4d220e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:43.243499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75aa72-0a82-48a8-9f02-17cb4e4d220e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:43.243811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75aa72-0a82-48a8-9f02-17cb4e4d220e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:43.244274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75aa72-0a82-48a8-9f02-17cb4e4d220e None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:43.245869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75aa72-0a82-48a8-9f02-17cb4e4d220e None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:43.248006 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75aa72-0a82-48a8-9f02-17cb4e4d220e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:43.255125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 109/233] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:43 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:43.262889 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80a162-2891-46c0-94f5-ad3d13933087 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:43.464418 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80a162-2891-46c0-94f5-ad3d13933087 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:43.534086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d1ecd-7b82-4670-bb1a-1b4ef23debb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:43.535298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d1ecd-7b82-4670-bb1a-1b4ef23debb8 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:43.535754 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d1ecd-7b82-4670-bb1a-1b4ef23debb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:43.536260 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d1ecd-7b82-4670-bb1a-1b4ef23debb8 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68/users/2eda438a5a214b5ebec919f278e5489e/roles/5a989bb8c8a1465ca76622ffb504dee5` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:43.538861 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d1ecd-7b82-4670-bb1a-1b4ef23debb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=551f963b02f4411395171476c9cc2d68, user_id=2eda438a5a214b5ebec919f278e5489e, role_id=5a989bb8c8a1465ca76622ffb504dee5)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:43.540828 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d1ecd-7b82-4670-bb1a-1b4ef23debb8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:43.565092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 125/234] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:55:43 2022] PUT /identity/v3/projects/551f963b02f4411395171476c9cc2d68/users/2eda438a5a214b5ebec919f278e5489e/roles/5a989bb8c8a1465ca76622ffb504dee5 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:55:44.426586 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2fa07-f2b5-4571-a222-fd8b87a96445 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:44.427104 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2fa07-f2b5-4571-a222-fd8b87a96445 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:44.427104 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2fa07-f2b5-4571-a222-fd8b87a96445 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:44.428806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 110/235] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:44.434133 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93630b21-b462-41cc-af0e-ec062a568e24 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:44.434599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93630b21-b462-41cc-af0e-ec062a568e24 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:44.434961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93630b21-b462-41cc-af0e-ec062a568e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:44.461125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-93630b21-b462-41cc-af0e-ec062a568e24 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:44.462455 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-93630b21-b462-41cc-af0e-ec062a568e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:44.496408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:44.547097 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 126/236] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:44.572749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2fa07-f2b5-4571-a222-fd8b87a96445 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:44.668119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9d359-a8b2-4549-8d63-bec52a4ef44e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:44.668518 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9d359-a8b2-4549-8d63-bec52a4ef44e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:44.668894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9d359-a8b2-4549-8d63-bec52a4ef44e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:44.670757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 127/237] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:55:44.679565 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620ce2d-0271-4891-95d5-b71da68bda8a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:44.680008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620ce2d-0271-4891-95d5-b71da68bda8a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:44.680502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620ce2d-0271-4891-95d5-b71da68bda8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:44.714062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-2620ce2d-0271-4891-95d5-b71da68bda8a None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:44.716563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:44.717816 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2620ce2d-0271-4891-95d5-b71da68bda8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:44.732901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2fa07-f2b5-4571-a222-fd8b87a96445 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:44.771189 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79cc50-66f4-477d-b30a-605680affe46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:44.772356 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79cc50-66f4-477d-b30a-605680affe46 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:44.772465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79cc50-66f4-477d-b30a-605680affe46 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:44.772714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79cc50-66f4-477d-b30a-605680affe46 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:44.785918 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79cc50-66f4-477d-b30a-605680affe46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-3d79cc50-66f4-477d-b30a-605680affe46 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:44.794842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:44.795765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:44.796621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:44.796621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:44.796621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:44.796621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:44.796621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:44.796621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:44.796621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:44.796621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:44.796621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:44.796621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:44.796621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:44.796621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:44.796621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 111/238] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:44.802030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb1b1a-bf05-4f84-856b-1257deb0f6c7 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:44.802030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb1b1a-bf05-4f84-856b-1257deb0f6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:44.802030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb1b1a-bf05-4f84-856b-1257deb0f6c7 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:44.803696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 112/239] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:44.804665 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb1b1a-bf05-4f84-856b-1257deb0f6c7 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:44.813736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 128/240] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:44.815955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122ad63-84c6-48fc-9384-7d5cf530c18a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:44.816433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122ad63-84c6-48fc-9384-7d5cf530c18a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:44.816897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122ad63-84c6-48fc-9384-7d5cf530c18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:44.846018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e122ad63-84c6-48fc-9384-7d5cf530c18a None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:44.848408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e122ad63-84c6-48fc-9384-7d5cf530c18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:44.926896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb1b1a-bf05-4f84-856b-1257deb0f6c7 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:44.948762 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 129/241] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:44.950487 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122ad63-84c6-48fc-9384-7d5cf530c18a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:44.973430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:44.990318 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce64b5-b9ca-40e3-80ac-930b7e271817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:44.991003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce64b5-b9ca-40e3-80ac-930b7e271817 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:44.991284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce64b5-b9ca-40e3-80ac-930b7e271817 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:44.991609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce64b5-b9ca-40e3-80ac-930b7e271817 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:44.992406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce64b5-b9ca-40e3-80ac-930b7e271817 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:44.992764 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce64b5-b9ca-40e3-80ac-930b7e271817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:44.994236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce64b5-b9ca-40e3-80ac-930b7e271817 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:45.007860 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 113/242] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:45.012172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce64b5-b9ca-40e3-80ac-930b7e271817 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:45.027280 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:45.090506 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122ad63-84c6-48fc-9384-7d5cf530c18a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:45.137688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d341c-52a8-4f27-9e1f-7d3041409b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:45.138933 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d341c-52a8-4f27-9e1f-7d3041409b5f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:45.139401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d341c-52a8-4f27-9e1f-7d3041409b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:45.139866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d341c-52a8-4f27-9e1f-7d3041409b5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:45.142385 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d341c-52a8-4f27-9e1f-7d3041409b5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:45.144423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d341c-52a8-4f27-9e1f-7d3041409b5f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:45.147324 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 130/243] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:55:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:45.153479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d341c-52a8-4f27-9e1f-7d3041409b5f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:45.191896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce64b5-b9ca-40e3-80ac-930b7e271817 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:45.245436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f1015-3cd1-48cb-9720-4c6459ba448f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:45.246626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f1015-3cd1-48cb-9720-4c6459ba448f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:45.247124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f1015-3cd1-48cb-9720-4c6459ba448f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:45.247877 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f1015-3cd1-48cb-9720-4c6459ba448f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:45.266095 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f1015-3cd1-48cb-9720-4c6459ba448f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:45.271925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-773f1015-3cd1-48cb-9720-4c6459ba448f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:45.279957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:45.280911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:45.281693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:45.281693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:45.281693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:45.281693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:45.281693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:45.281693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:45.281693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:45.281693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:45.281693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:45.281693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:45.281693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:55:45.281693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:45.284541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 114/244] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:55:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:45.285820 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f1015-3cd1-48cb-9720-4c6459ba448f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:45.297807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d341c-52a8-4f27-9e1f-7d3041409b5f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:45.357956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503a75f-0b28-46e9-a382-eb6c58d459b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:45.359184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503a75f-0b28-46e9-a382-eb6c58d459b9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:45.359617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503a75f-0b28-46e9-a382-eb6c58d459b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:45.360102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503a75f-0b28-46e9-a382-eb6c58d459b9 None admin] PATH_INFO: `/v3/users/de475cb8d1634c3b93391c945297f483` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:45.399313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503a75f-0b28-46e9-a382-eb6c58d459b9 None admin] RBAC: Authorizing `identity:get_user(user_id=de475cb8d1634c3b93391c945297f483)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:45.401118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503a75f-0b28-46e9-a382-eb6c58d459b9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:45.403995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 131/245] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:45 2022] GET /identity/v3/users/de475cb8d1634c3b93391c945297f483 => generated 297 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:45.404719 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503a75f-0b28-46e9-a382-eb6c58d459b9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:45.414570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:45.466667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f1015-3cd1-48cb-9720-4c6459ba448f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:45.517272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc931d94-a399-4b1f-9378-ac477088179f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:45.518381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc931d94-a399-4b1f-9378-ac477088179f None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:45.518812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc931d94-a399-4b1f-9378-ac477088179f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:45.519667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc931d94-a399-4b1f-9378-ac477088179f None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:45.520890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc931d94-a399-4b1f-9378-ac477088179f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:45.522724 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc931d94-a399-4b1f-9378-ac477088179f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:45.540024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503a75f-0b28-46e9-a382-eb6c58d459b9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:45.565941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 115/246] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:55:45 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:55:45.566533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc931d94-a399-4b1f-9378-ac477088179f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:45.597153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b2307-9067-4cd2-ac6e-4d6470fccfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:45.598879 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b2307-9067-4cd2-ac6e-4d6470fccfb3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:45.599579 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b2307-9067-4cd2-ac6e-4d6470fccfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:45.600243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b2307-9067-4cd2-ac6e-4d6470fccfb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:45.601845 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b2307-9067-4cd2-ac6e-4d6470fccfb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:45.602936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b2307-9067-4cd2-ac6e-4d6470fccfb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:45.605285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b2307-9067-4cd2-ac6e-4d6470fccfb3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:45.618500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 132/247] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:45.632564 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b2307-9067-4cd2-ac6e-4d6470fccfb3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:45.676772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:45.862491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc931d94-a399-4b1f-9378-ac477088179f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:45.898441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:45.921625 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdf8cf-fe6d-42e2-acc9-78e91b420fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:45.922416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdf8cf-fe6d-42e2-acc9-78e91b420fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:45.922672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdf8cf-fe6d-42e2-acc9-78e91b420fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:45.922948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdf8cf-fe6d-42e2-acc9-78e91b420fc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:45.939890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b2307-9067-4cd2-ac6e-4d6470fccfb3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:45.991986 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4150e35-60bf-4312-b29e-9ae501ecfa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:45.993259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4150e35-60bf-4312-b29e-9ae501ecfa2b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:45.993745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4150e35-60bf-4312-b29e-9ae501ecfa2b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:45.994216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4150e35-60bf-4312-b29e-9ae501ecfa2b None admin] PATH_INFO: `/v3/users/2eda438a5a214b5ebec919f278e5489e` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:45.996686 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4150e35-60bf-4312-b29e-9ae501ecfa2b None admin] RBAC: Authorizing `identity:get_user(user_id=2eda438a5a214b5ebec919f278e5489e)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:45.998203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4150e35-60bf-4312-b29e-9ae501ecfa2b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:46.000176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 133/248] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:45 2022] GET /identity/v3/users/2eda438a5a214b5ebec919f278e5489e => generated 283 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:46.001323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df442db7-d0db-4bed-8865-df5e1e4d44d7 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:46.001610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df442db7-d0db-4bed-8865-df5e1e4d44d7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:46.002085 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df442db7-d0db-4bed-8865-df5e1e4d44d7 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:46.005063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 134/249] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:46.008082 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df442db7-d0db-4bed-8865-df5e1e4d44d7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:46.057039 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacdf8cf-fe6d-42e2-acc9-78e91b420fc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:46.059575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacdf8cf-fe6d-42e2-acc9-78e91b420fc6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:46.136753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:46.180214 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df442db7-d0db-4bed-8865-df5e1e4d44d7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:46.241062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba8e64-bcaa-47a4-85f0-fd98e28049cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:46.241766 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba8e64-bcaa-47a4-85f0-fd98e28049cc None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:46.242097 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba8e64-bcaa-47a4-85f0-fd98e28049cc None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:46.242556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba8e64-bcaa-47a4-85f0-fd98e28049cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:46.282218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 116/250] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 716 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:46.284188 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bf075-46db-4445-a250-5a1af6a9cec0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:46.285126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bf075-46db-4445-a250-5a1af6a9cec0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:46.285644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bf075-46db-4445-a250-5a1af6a9cec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:46.314587 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-540bf075-46db-4445-a250-5a1af6a9cec0 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:46.316728 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-540bf075-46db-4445-a250-5a1af6a9cec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:46.350732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:46.402139 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba8e64-bcaa-47a4-85f0-fd98e28049cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:46.405027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba8e64-bcaa-47a4-85f0-fd98e28049cc None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:46.477673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 117/251] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:46.481144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bf075-46db-4445-a250-5a1af6a9cec0 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:46.533904 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:46.569092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 135/252] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:46.570048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba8e64-bcaa-47a4-85f0-fd98e28049cc None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:46.722286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bf075-46db-4445-a250-5a1af6a9cec0 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:46.739804 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba8e64-bcaa-47a4-85f0-fd98e28049cc None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:46.780503 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531c197-9d5f-44a7-b89c-fe0296451f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:46.781788 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531c197-9d5f-44a7-b89c-fe0296451f08 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:46.782329 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531c197-9d5f-44a7-b89c-fe0296451f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:46.782803 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531c197-9d5f-44a7-b89c-fe0296451f08 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:46.784145 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cc12b-840b-439a-869b-4379a474bbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:46.784975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cc12b-840b-439a-869b-4379a474bbbf None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:46.785176 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531c197-9d5f-44a7-b89c-fe0296451f08 None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:46.785274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cc12b-840b-439a-869b-4379a474bbbf None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:46.785522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cc12b-840b-439a-869b-4379a474bbbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:46.787310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531c197-9d5f-44a7-b89c-fe0296451f08 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:46.790372 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 118/253] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:46 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:46.793521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531c197-9d5f-44a7-b89c-fe0296451f08 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:46.803872 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cc12b-840b-439a-869b-4379a474bbbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-810cc12b-840b-439a-869b-4379a474bbbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:46.812707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:46.813707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:46.814554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:46.814554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:46.814554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:46.814554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:46.814554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:46.814554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:46.814554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:46.814554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:46.814554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:46.814554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:46.814554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:46.814554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:46.818987 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 136/254] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:46.820562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbe029-776c-4e10-9a39-0365bcc7a5a9 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:46.820860 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbe029-776c-4e10-9a39-0365bcc7a5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:46.821141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbe029-776c-4e10-9a39-0365bcc7a5a9 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:46.822624 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 137/255] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:46.825810 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbe029-776c-4e10-9a39-0365bcc7a5a9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:46.839817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:46.882309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:46.956561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531c197-9d5f-44a7-b89c-fe0296451f08 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:47.001870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d59c7-ee3a-486d-8b3a-249d3a1ab2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:47.002659 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d59c7-ee3a-486d-8b3a-249d3a1ab2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:47.002984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d59c7-ee3a-486d-8b3a-249d3a1ab2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:47.003268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d59c7-ee3a-486d-8b3a-249d3a1ab2e5 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:47.004554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d59c7-ee3a-486d-8b3a-249d3a1ab2e5 None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:47.006106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d59c7-ee3a-486d-8b3a-249d3a1ab2e5 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:47.008211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 119/256] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:46 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:47.009166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d59c7-ee3a-486d-8b3a-249d3a1ab2e5 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:47.009774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbe029-776c-4e10-9a39-0365bcc7a5a9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:47.068774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c926a-71e4-46bf-96d4-eb6781aa31af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:47.070472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c926a-71e4-46bf-96d4-eb6781aa31af None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:47.070992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c926a-71e4-46bf-96d4-eb6781aa31af None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:47.071472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c926a-71e4-46bf-96d4-eb6781aa31af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:47.072555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c926a-71e4-46bf-96d4-eb6781aa31af None admin] RBAC: Adding query filter params (role.id=0a6e69b6ee9e43659f71a5ba3260f5c9, scope.project.id=551f963b02f4411395171476c9cc2d68, user.id=de475cb8d1634c3b93391c945297f483) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:47.074600 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c926a-71e4-46bf-96d4-eb6781aa31af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:47.076644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c926a-71e4-46bf-96d4-eb6781aa31af None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:47.086841 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 138/257] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:46 2022] GET /identity/v3/role_assignments?user.id=de475cb8d1634c3b93391c945297f483&scope.project.id=551f963b02f4411395171476c9cc2d68&role.id=0a6e69b6ee9e43659f71a5ba3260f5c9 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:47.089047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198b3e0-1890-43a9-a6f7-df6e596a4172 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:47.089675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198b3e0-1890-43a9-a6f7-df6e596a4172 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:47.090911 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198b3e0-1890-43a9-a6f7-df6e596a4172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:47.113235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0198b3e0-1890-43a9-a6f7-df6e596a4172 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:47.114825 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0198b3e0-1890-43a9-a6f7-df6e596a4172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:47.131589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:47.160831 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d59c7-ee3a-486d-8b3a-249d3a1ab2e5 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:47.245920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7044d-0dc7-4f10-9998-eaf707c524f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:47.246633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7044d-0dc7-4f10-9998-eaf707c524f8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:47.246922 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7044d-0dc7-4f10-9998-eaf707c524f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:47.247198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7044d-0dc7-4f10-9998-eaf707c524f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:47.248013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7044d-0dc7-4f10-9998-eaf707c524f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:47.248328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7044d-0dc7-4f10-9998-eaf707c524f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:47.249811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7044d-0dc7-4f10-9998-eaf707c524f8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:47.263383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 139/258] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:47.264790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198b3e0-1890-43a9-a6f7-df6e596a4172 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:47.280441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 120/259] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:47.293656 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7044d-0dc7-4f10-9998-eaf707c524f8 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:47.330450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:47.426661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7044d-0dc7-4f10-9998-eaf707c524f8 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:47.470177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198b3e0-1890-43a9-a6f7-df6e596a4172 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:47.481675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae7fbf-a55d-41e5-a2d2-ba8390505477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:47.482576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae7fbf-a55d-41e5-a2d2-ba8390505477 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:47.482861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae7fbf-a55d-41e5-a2d2-ba8390505477 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:47.483146 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae7fbf-a55d-41e5-a2d2-ba8390505477 None admin] PATH_INFO: `/v3/users/860b4fd63ee04b2fa8fc1a0c33b66ecc` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:47.485373 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae7fbf-a55d-41e5-a2d2-ba8390505477 None admin] RBAC: Authorizing `identity:get_user(user_id=860b4fd63ee04b2fa8fc1a0c33b66ecc)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:47.486871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae7fbf-a55d-41e5-a2d2-ba8390505477 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:47.488672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 121/260] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:47 2022] GET /identity/v3/users/860b4fd63ee04b2fa8fc1a0c33b66ecc => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:47.494321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae7fbf-a55d-41e5-a2d2-ba8390505477 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:47.534253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754e249-d828-4b14-8e74-47d81dce0aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:47.535313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754e249-d828-4b14-8e74-47d81dce0aae None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:47.535788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754e249-d828-4b14-8e74-47d81dce0aae None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:47.536325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754e249-d828-4b14-8e74-47d81dce0aae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:47.537889 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754e249-d828-4b14-8e74-47d81dce0aae None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=551f963b02f4411395171476c9cc2d68, user.id=2eda438a5a214b5ebec919f278e5489e) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:47.540662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754e249-d828-4b14-8e74-47d81dce0aae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:47.548623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754e249-d828-4b14-8e74-47d81dce0aae None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:47.553470 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:47.575537 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 140/261] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:47 2022] GET /identity/v3/role_assignments?user.id=2eda438a5a214b5ebec919f278e5489e&scope.project.id=551f963b02f4411395171476c9cc2d68&role.id=000e942517f2469ba36d2e65978bc314 => generated 608 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:47.576998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754e249-d828-4b14-8e74-47d81dce0aae None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:47.790020 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae7fbf-a55d-41e5-a2d2-ba8390505477 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:47.844047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:47.868895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754e249-d828-4b14-8e74-47d81dce0aae None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:47.881795 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15471fa-c8a7-4d24-89e9-487a67fa5321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:47.882611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15471fa-c8a7-4d24-89e9-487a67fa5321 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:47.882891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15471fa-c8a7-4d24-89e9-487a67fa5321 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:47.883168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15471fa-c8a7-4d24-89e9-487a67fa5321 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:47.907046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15471fa-c8a7-4d24-89e9-487a67fa5321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b15471fa-c8a7-4d24-89e9-487a67fa5321 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:47.918542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:47.919701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:47.920664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:47.920664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:47.920664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:47.920664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:47.920664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:47.920664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:47.920664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:47.920664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:47.920664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:47.920664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:47.920664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:55:47.920664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:47.920664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 122/262] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:55:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:47.931948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15471fa-c8a7-4d24-89e9-487a67fa5321 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:47.956953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c341af5-c0e2-46b8-8dfe-a05f95d0dc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:47.958333 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c341af5-c0e2-46b8-8dfe-a05f95d0dc75 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:47.958832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c341af5-c0e2-46b8-8dfe-a05f95d0dc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:47.959331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c341af5-c0e2-46b8-8dfe-a05f95d0dc75 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:47.961129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c341af5-c0e2-46b8-8dfe-a05f95d0dc75 None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:47.963049 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c341af5-c0e2-46b8-8dfe-a05f95d0dc75 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:47.965802 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 141/263] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:47 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:47.975466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c341af5-c0e2-46b8-8dfe-a05f95d0dc75 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:48.121929 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c341af5-c0e2-46b8-8dfe-a05f95d0dc75 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:48.157249 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:48.168501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c120e-d831-4e22-a5b3-1426bd0a2168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:48.170720 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c120e-d831-4e22-a5b3-1426bd0a2168 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:48.171241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c120e-d831-4e22-a5b3-1426bd0a2168 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:48.171794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c120e-d831-4e22-a5b3-1426bd0a2168 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68/users/860b4fd63ee04b2fa8fc1a0c33b66ecc/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:48.176237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c120e-d831-4e22-a5b3-1426bd0a2168 None admin] RBAC: Authorizing `identity:create_grant(project_id=551f963b02f4411395171476c9cc2d68, user_id=860b4fd63ee04b2fa8fc1a0c33b66ecc, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:48.179268 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c120e-d831-4e22-a5b3-1426bd0a2168 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:48.206701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15471fa-c8a7-4d24-89e9-487a67fa5321 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:48.206972 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 142/264] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:55:47 2022] PUT /identity/v3/projects/551f963b02f4411395171476c9cc2d68/users/860b4fd63ee04b2fa8fc1a0c33b66ecc/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:55:48.280877 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d22dfb-074e-49c3-a35a-ebb7bceea0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:48.281603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d22dfb-074e-49c3-a35a-ebb7bceea0bf None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:48.281893 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d22dfb-074e-49c3-a35a-ebb7bceea0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:48.282186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d22dfb-074e-49c3-a35a-ebb7bceea0bf None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:48.283019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d22dfb-074e-49c3-a35a-ebb7bceea0bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:48.283339 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d22dfb-074e-49c3-a35a-ebb7bceea0bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:48.284895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d22dfb-074e-49c3-a35a-ebb7bceea0bf None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:48.295817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 123/265] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:55:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:48.301931 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c120e-d831-4e22-a5b3-1426bd0a2168 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:48.343253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:48.454118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c120e-d831-4e22-a5b3-1426bd0a2168 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:48.496645 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596bc6ac-0324-467c-9a79-9dbac4304d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:48.498191 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bc6ac-0324-467c-9a79-9dbac4304d6a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:48.498689 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bc6ac-0324-467c-9a79-9dbac4304d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:48.499176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596bc6ac-0324-467c-9a79-9dbac4304d6a None admin] PATH_INFO: `/v3/users/2eda438a5a214b5ebec919f278e5489e` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:48.501463 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596bc6ac-0324-467c-9a79-9dbac4304d6a None admin] RBAC: Authorizing `identity:get_user(user_id=2eda438a5a214b5ebec919f278e5489e)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:48.504241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596bc6ac-0324-467c-9a79-9dbac4304d6a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:48.507109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 143/266] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:48 2022] GET /identity/v3/users/2eda438a5a214b5ebec919f278e5489e => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:48.513234 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d22dfb-074e-49c3-a35a-ebb7bceea0bf None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:48.688557 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d22dfb-074e-49c3-a35a-ebb7bceea0bf None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:48.770405 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af615a-4254-45d2-8c37-696351ba0c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:48.771516 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af615a-4254-45d2-8c37-696351ba0c00 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:48.772049 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af615a-4254-45d2-8c37-696351ba0c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:48.772535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af615a-4254-45d2-8c37-696351ba0c00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:48.806501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:48.908835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af615a-4254-45d2-8c37-696351ba0c00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:48.911396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af615a-4254-45d2-8c37-696351ba0c00 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:48.957025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6696d46-4fed-4a3c-9b86-f5a3b265ab96 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:48.957574 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6696d46-4fed-4a3c-9b86-f5a3b265ab96 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:48.957780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6696d46-4fed-4a3c-9b86-f5a3b265ab96 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:48.959770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 144/267] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:48.966793 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2666e6-7701-4e0b-b902-dc28ccebc7be None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:48.967408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2666e6-7701-4e0b-b902-dc28ccebc7be None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:48.967899 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2666e6-7701-4e0b-b902-dc28ccebc7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:49.005199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ad2666e6-7701-4e0b-b902-dc28ccebc7be None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:49.006664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ad2666e6-7701-4e0b-b902-dc28ccebc7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:49.119212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 145/268] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:49.121010 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a17a02-f4b8-4042-a2a2-14e72623fbee None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:49.121299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a17a02-f4b8-4042-a2a2-14e72623fbee None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:49.121571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a17a02-f4b8-4042-a2a2-14e72623fbee None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:49.123176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 146/269] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:49.128679 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4220f3-00a2-4037-9450-baa15740634d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:49.129060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4220f3-00a2-4037-9450-baa15740634d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:49.129303 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4220f3-00a2-4037-9450-baa15740634d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:49.157477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 124/270] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:49.159439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af615a-4254-45d2-8c37-696351ba0c00 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:49.161003 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7d4220f3-00a2-4037-9450-baa15740634d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:49.162307 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7d4220f3-00a2-4037-9450-baa15740634d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:49.178827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:49.219947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:49.239631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 147/271] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:55:49.240543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4220f3-00a2-4037-9450-baa15740634d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:49.283608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af615a-4254-45d2-8c37-696351ba0c00 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:49.324342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce732147-b642-43ad-a29f-5031ab3520c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:49.325264 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce732147-b642-43ad-a29f-5031ab3520c4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:49.325547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce732147-b642-43ad-a29f-5031ab3520c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:49.325821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce732147-b642-43ad-a29f-5031ab3520c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:49.327077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce732147-b642-43ad-a29f-5031ab3520c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:49.328789 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce732147-b642-43ad-a29f-5031ab3520c4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:49.330851 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 125/272] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:55:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:49.331411 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce732147-b642-43ad-a29f-5031ab3520c4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:49.383785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4220f3-00a2-4037-9450-baa15740634d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:49.423650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fda9b-8e43-49c2-957f-dd545d68c012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:49.424569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fda9b-8e43-49c2-957f-dd545d68c012 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:49.424856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fda9b-8e43-49c2-957f-dd545d68c012 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:49.425108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fda9b-8e43-49c2-957f-dd545d68c012 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:49.426446 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fda9b-8e43-49c2-957f-dd545d68c012 None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:49.428031 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fda9b-8e43-49c2-957f-dd545d68c012 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:49.429878 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 148/273] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:49 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:49.430657 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fda9b-8e43-49c2-957f-dd545d68c012 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:49.458451 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce732147-b642-43ad-a29f-5031ab3520c4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:49.462348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:49.476275 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:49.501743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e397ec-bf0d-4df9-91e1-ce4b367d3c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:49.502843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e397ec-bf0d-4df9-91e1-ce4b367d3c77 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:49.503320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e397ec-bf0d-4df9-91e1-ce4b367d3c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:49.503775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e397ec-bf0d-4df9-91e1-ce4b367d3c77 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:49.516347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e397ec-bf0d-4df9-91e1-ce4b367d3c77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-c4e397ec-bf0d-4df9-91e1-ce4b367d3c77 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:49.525225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:49.526173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:49.527242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:49.527242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:49.527242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:49.527242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:49.527242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:49.527242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:49.527242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:49.527242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:49.527242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:49.527242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:49.527242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:49.527242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:49.530098 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 126/274] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:49.531114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e397ec-bf0d-4df9-91e1-ce4b367d3c77 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:49.628119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fda9b-8e43-49c2-957f-dd545d68c012 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:49.644388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e397ec-bf0d-4df9-91e1-ce4b367d3c77 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:49.683993 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b8eba-2193-4ed6-8eb3-a75c625a90ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:49.685098 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b8eba-2193-4ed6-8eb3-a75c625a90ab None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:49.685560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b8eba-2193-4ed6-8eb3-a75c625a90ab None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:49.686101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b8eba-2193-4ed6-8eb3-a75c625a90ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:49.687340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b8eba-2193-4ed6-8eb3-a75c625a90ab None admin] RBAC: Adding query filter params (role.id=5a989bb8c8a1465ca76622ffb504dee5, scope.project.id=551f963b02f4411395171476c9cc2d68, user.id=2eda438a5a214b5ebec919f278e5489e) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:49.687954 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b8eba-2193-4ed6-8eb3-a75c625a90ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:49.690011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b8eba-2193-4ed6-8eb3-a75c625a90ab None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:49.690539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed8766-2984-4c3d-872b-e7052c60259f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:49.691426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed8766-2984-4c3d-872b-e7052c60259f None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:49.691714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed8766-2984-4c3d-872b-e7052c60259f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:49.692027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed8766-2984-4c3d-872b-e7052c60259f None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:49.692877 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed8766-2984-4c3d-872b-e7052c60259f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:49.694973 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed8766-2984-4c3d-872b-e7052c60259f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:49.699223 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 127/275] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:49 2022] GET /identity/v3/role_assignments?user.id=2eda438a5a214b5ebec919f278e5489e&scope.project.id=551f963b02f4411395171476c9cc2d68&role.id=5a989bb8c8a1465ca76622ffb504dee5 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:49.700438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b8eba-2193-4ed6-8eb3-a75c625a90ab None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:49.748631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 149/276] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:55:49 2022] POST /identity/v3/users => generated 293 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:55:49.749023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:49.880440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b8eba-2193-4ed6-8eb3-a75c625a90ab None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:49.952880 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e79b1d-9e07-413f-9056-45a122c5e02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:49.953594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e79b1d-9e07-413f-9056-45a122c5e02b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:49.953878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e79b1d-9e07-413f-9056-45a122c5e02b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:49.954181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e79b1d-9e07-413f-9056-45a122c5e02b None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:49.954971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e79b1d-9e07-413f-9056-45a122c5e02b None admin] RBAC: Adding query filter params (name=member) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:49.955287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e79b1d-9e07-413f-9056-45a122c5e02b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:49.958093 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e79b1d-9e07-413f-9056-45a122c5e02b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:49.968799 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 128/277] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:49.975276 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed8766-2984-4c3d-872b-e7052c60259f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:50.135399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:50.186465 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed8766-2984-4c3d-872b-e7052c60259f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:50.241555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c0ee33-3db2-41fb-8d95-85a3a72b14a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:50.243083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0ee33-3db2-41fb-8d95-85a3a72b14a4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:50.243568 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0ee33-3db2-41fb-8d95-85a3a72b14a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:50.244106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c0ee33-3db2-41fb-8d95-85a3a72b14a4 None admin] PATH_INFO: `/v3/users/805ff5eabce744d2bc6b22ca4a6b3743` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:50.269943 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0ee33-3db2-41fb-8d95-85a3a72b14a4 None admin] RBAC: Authorizing `identity:get_user(user_id=805ff5eabce744d2bc6b22ca4a6b3743)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:50.271531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c0ee33-3db2-41fb-8d95-85a3a72b14a4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:50.273937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 150/278] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:49 2022] GET /identity/v3/users/805ff5eabce744d2bc6b22ca4a6b3743 => generated 293 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:50.280355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e79b1d-9e07-413f-9056-45a122c5e02b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:50.429661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f31de-e43b-45b8-b94c-db64f67c0128 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:50.429951 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f31de-e43b-45b8-b94c-db64f67c0128 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:50.430353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f31de-e43b-45b8-b94c-db64f67c0128 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:50.431817 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 151/279] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:55:50.438572 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0c85f-4502-4d09-85ad-91229be5f9d0 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:50.438898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0c85f-4502-4d09-85ad-91229be5f9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:50.439301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0c85f-4502-4d09-85ad-91229be5f9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:50.458825 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a5d0c85f-4502-4d09-85ad-91229be5f9d0 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:50.460213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a5d0c85f-4502-4d09-85ad-91229be5f9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:50.478619 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:50.486074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e79b1d-9e07-413f-9056-45a122c5e02b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:50.552577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e4dd3-ed73-403c-9450-cc0056fa8df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:50.554943 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e4dd3-ed73-403c-9450-cc0056fa8df6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:50.555419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 152/280] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:55:50.556051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e4dd3-ed73-403c-9450-cc0056fa8df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:50.556872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e4dd3-ed73-403c-9450-cc0056fa8df6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:50.558859 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e4dd3-ed73-403c-9450-cc0056fa8df6 None admin] RBAC: Adding query filter params (role.id=3d7c80cf23864f58bf21f893ba7691d6, user.id=805ff5eabce744d2bc6b22ca4a6b3743, scope.system=all) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:50.559679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e4dd3-ed73-403c-9450-cc0056fa8df6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:50.562756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e4dd3-ed73-403c-9450-cc0056fa8df6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:50.574554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 129/281] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Jul 21 08:55:50 2022] GET /identity/v3/role_assignments?user.id=805ff5eabce744d2bc6b22ca4a6b3743&scope.system=all&role.id=3d7c80cf23864f58bf21f893ba7691d6 => generated 226 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:50.593493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0c85f-4502-4d09-85ad-91229be5f9d0 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:50.708924 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:50.763048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0c85f-4502-4d09-85ad-91229be5f9d0 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:50.810867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c22dc-e5da-44d1-a82b-ee4450385d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:50.812153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c22dc-e5da-44d1-a82b-ee4450385d56 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:50.812640 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c22dc-e5da-44d1-a82b-ee4450385d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:50.813132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c22dc-e5da-44d1-a82b-ee4450385d56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:50.826664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c22dc-e5da-44d1-a82b-ee4450385d56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-327c22dc-e5da-44d1-a82b-ee4450385d56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:50.834985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:50.835962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:50.836753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:50.836753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:50.836753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:50.836753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:50.836753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:50.836753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:50.836753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:50.836753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:50.836753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:50.836753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:50.836753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:50.836753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:50.839419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 153/282] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:55:50.849427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e4dd3-ed73-403c-9450-cc0056fa8df6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:51.075899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e4dd3-ed73-403c-9450-cc0056fa8df6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:51.085765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b24dad-ab9e-48fa-9d6c-72f6aa17cc77 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:51.086743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b24dad-ab9e-48fa-9d6c-72f6aa17cc77 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:51.087235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b24dad-ab9e-48fa-9d6c-72f6aa17cc77 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:51.096547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 154/283] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:51 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:51.101867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c46be-cb37-4e91-9d06-80cbbab42ce4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:51.102457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c46be-cb37-4e91-9d06-80cbbab42ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:51.102937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c46be-cb37-4e91-9d06-80cbbab42ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:51.138682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76430c-b4b9-4882-ac3d-ef1e4cb2c708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:51.140180 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76430c-b4b9-4882-ac3d-ef1e4cb2c708 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:51.140852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76430c-b4b9-4882-ac3d-ef1e4cb2c708 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:51.141535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76430c-b4b9-4882-ac3d-ef1e4cb2c708 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:51.143173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76430c-b4b9-4882-ac3d-ef1e4cb2c708 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:51.143980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76430c-b4b9-4882-ac3d-ef1e4cb2c708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:51.146775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76430c-b4b9-4882-ac3d-ef1e4cb2c708 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:51.163770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 130/284] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:51.165901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-bf9c46be-cb37-4e91-9d06-80cbbab42ce4 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:51.167769 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-bf9c46be-cb37-4e91-9d06-80cbbab42ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:51.173229 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76430c-b4b9-4882-ac3d-ef1e4cb2c708 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:51.220325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:51.323622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 155/285] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:51.362045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76430c-b4b9-4882-ac3d-ef1e4cb2c708 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:51.380464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c46be-cb37-4e91-9d06-80cbbab42ce4 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:51.426120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a2dfd-6cc0-4502-a599-cd1758fbb6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:51.427096 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a2dfd-6cc0-4502-a599-cd1758fbb6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:51.427384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a2dfd-6cc0-4502-a599-cd1758fbb6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:51.427661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a2dfd-6cc0-4502-a599-cd1758fbb6b7 None admin] PATH_INFO: `/v3/users/de475cb8d1634c3b93391c945297f483` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:51.429021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a2dfd-6cc0-4502-a599-cd1758fbb6b7 None admin] RBAC: Authorizing `identity:get_user(user_id=de475cb8d1634c3b93391c945297f483)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:51.434668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a2dfd-6cc0-4502-a599-cd1758fbb6b7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:51.435198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 131/286] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:51 2022] GET /identity/v3/users/de475cb8d1634c3b93391c945297f483 => generated 297 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:51.441791 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a2dfd-6cc0-4502-a599-cd1758fbb6b7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:51.473381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:51.591030 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c46be-cb37-4e91-9d06-80cbbab42ce4 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:51.638899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:51.654742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a2dfd-6cc0-4502-a599-cd1758fbb6b7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:51.671989 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cbae1-8075-4b26-b681-2f596fd8f781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:51.672811 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cbae1-8075-4b26-b681-2f596fd8f781 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:51.673096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cbae1-8075-4b26-b681-2f596fd8f781 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:51.673374 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cbae1-8075-4b26-b681-2f596fd8f781 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:51.699750 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68cbae1-8075-4b26-b681-2f596fd8f781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-b68cbae1-8075-4b26-b681-2f596fd8f781 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:51.709186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:51.710144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:51.710927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:51.710927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:51.710927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:51.710927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:51.710927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:51.710927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:51.710927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:51.710927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:51.710927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:51.710927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:51.710927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:51.710927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:51.711518 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 156/287] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:51.719936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cbae1-8075-4b26-b681-2f596fd8f781 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:51.735083 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de311fd6-6b1e-4e2b-861e-fe856133a9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:51.736486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de311fd6-6b1e-4e2b-861e-fe856133a9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:51.736986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de311fd6-6b1e-4e2b-861e-fe856133a9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:51.737507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de311fd6-6b1e-4e2b-861e-fe856133a9f5 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:51.738972 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de311fd6-6b1e-4e2b-861e-fe856133a9f5 None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:51.740684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de311fd6-6b1e-4e2b-861e-fe856133a9f5 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:51.743310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 132/288] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:51 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:51.754043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de311fd6-6b1e-4e2b-861e-fe856133a9f5 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:51.939089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de311fd6-6b1e-4e2b-861e-fe856133a9f5 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:51.974104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cbae1-8075-4b26-b681-2f596fd8f781 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:52.002999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef306fde-e9b2-4326-8e76-a885ff658284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:52.004980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef306fde-e9b2-4326-8e76-a885ff658284 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:52.007117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef306fde-e9b2-4326-8e76-a885ff658284 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:52.009888 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef306fde-e9b2-4326-8e76-a885ff658284 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68/users/de475cb8d1634c3b93391c945297f483/roles/0a6e69b6ee9e43659f71a5ba3260f5c9` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:52.015199 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef306fde-e9b2-4326-8e76-a885ff658284 None admin] RBAC: Authorizing `identity:create_grant(project_id=551f963b02f4411395171476c9cc2d68, user_id=de475cb8d1634c3b93391c945297f483, role_id=0a6e69b6ee9e43659f71a5ba3260f5c9)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:52.016477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:52.017287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef306fde-e9b2-4326-8e76-a885ff658284 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:52.046555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 133/289] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:55:51 2022] PUT /identity/v3/projects/551f963b02f4411395171476c9cc2d68/users/de475cb8d1634c3b93391c945297f483/roles/0a6e69b6ee9e43659f71a5ba3260f5c9 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:55:52.077732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ff3bc-9047-43da-8d7f-4ad45a7ca094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:52.078830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ff3bc-9047-43da-8d7f-4ad45a7ca094 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:52.079323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ff3bc-9047-43da-8d7f-4ad45a7ca094 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:52.079843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ff3bc-9047-43da-8d7f-4ad45a7ca094 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:52.083478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ff3bc-9047-43da-8d7f-4ad45a7ca094 None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:52.084210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ff3bc-9047-43da-8d7f-4ad45a7ca094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:52.085906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ff3bc-9047-43da-8d7f-4ad45a7ca094 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:52.102019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 157/290] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:52.111678 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef306fde-e9b2-4326-8e76-a885ff658284 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:52.145197 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:52.264705 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef306fde-e9b2-4326-8e76-a885ff658284 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:52.343808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7f467-4b7a-4d58-93c0-6b10901f3449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:52.345427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7f467-4b7a-4d58-93c0-6b10901f3449 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:52.345938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7f467-4b7a-4d58-93c0-6b10901f3449 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:52.346446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7f467-4b7a-4d58-93c0-6b10901f3449 None admin] PATH_INFO: `/v3/users/860b4fd63ee04b2fa8fc1a0c33b66ecc` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:52.349914 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7f467-4b7a-4d58-93c0-6b10901f3449 None admin] RBAC: Authorizing `identity:get_user(user_id=860b4fd63ee04b2fa8fc1a0c33b66ecc)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:52.352827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7f467-4b7a-4d58-93c0-6b10901f3449 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:52.357592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 134/291] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:52 2022] GET /identity/v3/users/860b4fd63ee04b2fa8fc1a0c33b66ecc => generated 297 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:52.363746 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ff3bc-9047-43da-8d7f-4ad45a7ca094 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:52.494922 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ff3bc-9047-43da-8d7f-4ad45a7ca094 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:52.534133 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e3666-548c-42fd-9683-90bf7f3ea255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:52.535146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e3666-548c-42fd-9683-90bf7f3ea255 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:52.535655 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e3666-548c-42fd-9683-90bf7f3ea255 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:52.536171 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e3666-548c-42fd-9683-90bf7f3ea255 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:52.583939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:52.614006 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e3666-548c-42fd-9683-90bf7f3ea255 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:52.615521 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e3666-548c-42fd-9683-90bf7f3ea255 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:52.747981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 158/292] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:55:52.753181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7f467-4b7a-4d58-93c0-6b10901f3449 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:52.911063 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:52.929174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7f467-4b7a-4d58-93c0-6b10901f3449 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:52.971509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffb294-5ca2-437f-88a6-be8da8b61186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:52.972426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffb294-5ca2-437f-88a6-be8da8b61186 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:52.972716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffb294-5ca2-437f-88a6-be8da8b61186 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:52.973001 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffb294-5ca2-437f-88a6-be8da8b61186 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:52.974251 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ffb294-5ca2-437f-88a6-be8da8b61186 None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:52.975766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ffb294-5ca2-437f-88a6-be8da8b61186 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:52.977864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 135/293] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:52 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:52.987304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e3666-548c-42fd-9683-90bf7f3ea255 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:53.088935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:53.134456 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e3666-548c-42fd-9683-90bf7f3ea255 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:53.185792 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ffca1-986a-432a-99fd-78e5c8fefa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:53.186553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ffca1-986a-432a-99fd-78e5c8fefa38 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:53.186841 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ffca1-986a-432a-99fd-78e5c8fefa38 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:53.187113 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ffca1-986a-432a-99fd-78e5c8fefa38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:53.187982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ffca1-986a-432a-99fd-78e5c8fefa38 None admin] RBAC: Adding query filter params (role.id=3d7c80cf23864f58bf21f893ba7691d6, scope.project.id=551f963b02f4411395171476c9cc2d68, user.id=860b4fd63ee04b2fa8fc1a0c33b66ecc) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:53.188293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ffca1-986a-432a-99fd-78e5c8fefa38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:53.189746 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ffca1-986a-432a-99fd-78e5c8fefa38 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:53.201393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 159/294] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:52 2022] GET /identity/v3/role_assignments?user.id=860b4fd63ee04b2fa8fc1a0c33b66ecc&scope.project.id=551f963b02f4411395171476c9cc2d68&role.id=3d7c80cf23864f58bf21f893ba7691d6 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:53.242364 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb442d9-fb88-4c10-a727-bde939693ffe None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:53.242666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb442d9-fb88-4c10-a727-bde939693ffe None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:53.243144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb442d9-fb88-4c10-a727-bde939693ffe None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:53.245694 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 136/295] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:53.251024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad85a2-678f-49a3-8421-039d6751cbf6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:53.251311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad85a2-678f-49a3-8421-039d6751cbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:53.251590 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad85a2-678f-49a3-8421-039d6751cbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:53.279315 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c6ad85a2-678f-49a3-8421-039d6751cbf6 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:53.281383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c6ad85a2-678f-49a3-8421-039d6751cbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:53.390176 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b3352-0fa7-4900-bbc5-144f1a885001 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:53.390854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b3352-0fa7-4900-bbc5-144f1a885001 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:53.391403 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b3352-0fa7-4900-bbc5-144f1a885001 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:53.393124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 137/296] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:53.411842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c2c1a-6ada-4daa-a064-cb4f95f20d1d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:53.417126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c2c1a-6ada-4daa-a064-cb4f95f20d1d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:53.418389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c2c1a-6ada-4daa-a064-cb4f95f20d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:53.419368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 160/297] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:53.445560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad85a2-678f-49a3-8421-039d6751cbf6 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:53.473319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ec1c2c1a-6ada-4daa-a064-cb4f95f20d1d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:53.475164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ec1c2c1a-6ada-4daa-a064-cb4f95f20d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:53.537983 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:53.590841 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad85a2-678f-49a3-8421-039d6751cbf6 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:53.643256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 138/298] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:53.646203 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4fa7a-cb17-4574-990c-f669c1b04aa5 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:53.646706 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4fa7a-cb17-4574-990c-f669c1b04aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:53.647544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4fa7a-cb17-4574-990c-f669c1b04aa5 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:53.651095 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafdd6ac-c81e-44a9-8b2c-b0b7f4fa5cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:53.652377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdd6ac-c81e-44a9-8b2c-b0b7f4fa5cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:53.652902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdd6ac-c81e-44a9-8b2c-b0b7f4fa5cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:53.653313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafdd6ac-c81e-44a9-8b2c-b0b7f4fa5cd2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:53.656459 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 139/299] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:53.660687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4837e-a6b2-4aa1-ad38-293031c0314c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:53.661562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4837e-a6b2-4aa1-ad38-293031c0314c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:53.662318 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4837e-a6b2-4aa1-ad38-293031c0314c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:53.669168 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafdd6ac-c81e-44a9-8b2c-b0b7f4fa5cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-bafdd6ac-c81e-44a9-8b2c-b0b7f4fa5cd2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:53.686541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:53.688389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:53.690024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:53.690024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:53.690024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:53.690024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:53.690024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:53.690024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:53.690024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:53.690024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:53.690024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:53.690024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:53.690024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:53.690024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:53.690024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 161/300] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:55:53.690024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafdd6ac-c81e-44a9-8b2c-b0b7f4fa5cd2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:53.699374 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-83b4837e-a6b2-4aa1-ad38-293031c0314c None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:53.701617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-83b4837e-a6b2-4aa1-ad38-293031c0314c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:53.832305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 140/301] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:53.834816 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4837e-a6b2-4aa1-ad38-293031c0314c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:53.845251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafdd6ac-c81e-44a9-8b2c-b0b7f4fa5cd2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:53.869045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:53.890045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:53.908406 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbb55a-141c-49dd-9ed6-b350efefe364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:53.909810 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbb55a-141c-49dd-9ed6-b350efefe364 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:53.910390 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbb55a-141c-49dd-9ed6-b350efefe364 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:53.910869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbb55a-141c-49dd-9ed6-b350efefe364 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:53.912558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbb55a-141c-49dd-9ed6-b350efefe364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:53.915251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbb55a-141c-49dd-9ed6-b350efefe364 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:53.922642 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 162/302] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:55:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:53.923514 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbb55a-141c-49dd-9ed6-b350efefe364 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:53.998407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4837e-a6b2-4aa1-ad38-293031c0314c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:54.042725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffb516-292f-4dc4-ad87-3bf8af046725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:54.043433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb516-292f-4dc4-ad87-3bf8af046725 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:54.043722 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb516-292f-4dc4-ad87-3bf8af046725 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:54.044282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb516-292f-4dc4-ad87-3bf8af046725 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:54.045261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffb516-292f-4dc4-ad87-3bf8af046725 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:54.045574 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffb516-292f-4dc4-ad87-3bf8af046725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:54.047705 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffb516-292f-4dc4-ad87-3bf8af046725 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:54.067561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 141/303] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:54.067561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffb516-292f-4dc4-ad87-3bf8af046725 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:54.115266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbb55a-141c-49dd-9ed6-b350efefe364 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:54.164422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715fd127-fa57-4d99-8cb9-691bebb5b380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:54.166710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fd127-fa57-4d99-8cb9-691bebb5b380 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:54.166710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fd127-fa57-4d99-8cb9-691bebb5b380 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:54.166710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fd127-fa57-4d99-8cb9-691bebb5b380 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:54.181588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715fd127-fa57-4d99-8cb9-691bebb5b380 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-715fd127-fa57-4d99-8cb9-691bebb5b380 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:54.190945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:54.192881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:54.194086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:54.194086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:54.194086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:54.194086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:54.194086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:54.194086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:54.194086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:54.194086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:54.194086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:54.194086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:54.194086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:54.194086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:54.194086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 163/304] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:54.196180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715fd127-fa57-4d99-8cb9-691bebb5b380 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:54.196535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:54.226317 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffb516-292f-4dc4-ad87-3bf8af046725 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:54.268592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:54.273648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830f2ec-8f68-4460-a252-3f19d2f376b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:54.274725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830f2ec-8f68-4460-a252-3f19d2f376b7 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:54.275158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830f2ec-8f68-4460-a252-3f19d2f376b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:54.275595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830f2ec-8f68-4460-a252-3f19d2f376b7 None admin] PATH_INFO: `/v3/groups` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:54.277236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830f2ec-8f68-4460-a252-3f19d2f376b7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:54.279711 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830f2ec-8f68-4460-a252-3f19d2f376b7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:54.297393 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 142/305] 10.222.0.13 () {60 vars in 1134 bytes} [Thu Jul 21 08:55:54 2022] POST /identity/v3/groups => generated 229 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:55:54.308046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830f2ec-8f68-4460-a252-3f19d2f376b7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:54.345406 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715fd127-fa57-4d99-8cb9-691bebb5b380 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:54.394582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98c730-e076-430c-91a0-5ea86268888f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:54.395988 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98c730-e076-430c-91a0-5ea86268888f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:54.396253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98c730-e076-430c-91a0-5ea86268888f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:54.396537 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98c730-e076-430c-91a0-5ea86268888f None admin] PATH_INFO: `/v3/users/b1c9ebb579ed4f2983b1ef77c59be8f5` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:54.428956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb98c730-e076-430c-91a0-5ea86268888f None admin] RBAC: Authorizing `identity:get_user(user_id=b1c9ebb579ed4f2983b1ef77c59be8f5)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:54.432254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb98c730-e076-430c-91a0-5ea86268888f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:54.436851 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 164/306] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:54 2022] GET /identity/v3/users/b1c9ebb579ed4f2983b1ef77c59be8f5 => generated 293 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:54.446744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98c730-e076-430c-91a0-5ea86268888f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:54.454556 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830f2ec-8f68-4460-a252-3f19d2f376b7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:54.502699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564435ab-0e47-4afd-8c0d-9fbf5bde5ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:54.503384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564435ab-0e47-4afd-8c0d-9fbf5bde5ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:54.503664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564435ab-0e47-4afd-8c0d-9fbf5bde5ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:54.503957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564435ab-0e47-4afd-8c0d-9fbf5bde5ad1 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:54.505201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564435ab-0e47-4afd-8c0d-9fbf5bde5ad1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:54.505310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564435ab-0e47-4afd-8c0d-9fbf5bde5ad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:54.506705 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564435ab-0e47-4afd-8c0d-9fbf5bde5ad1 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:54.513883 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:54.524770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 143/307] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:54.527350 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66967c-e61c-46c0-b204-df80a0aa7dbb None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:54.527943 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66967c-e61c-46c0-b204-df80a0aa7dbb None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:54.531522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66967c-e61c-46c0-b204-df80a0aa7dbb None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:54.533259 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 144/308] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:54.537119 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66967c-e61c-46c0-b204-df80a0aa7dbb None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:54.613893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98c730-e076-430c-91a0-5ea86268888f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:54.639368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:54.671875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66967c-e61c-46c0-b204-df80a0aa7dbb None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:54.677256 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8074e-e559-4954-aff5-19a98dab521b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:54.678098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8074e-e559-4954-aff5-19a98dab521b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:54.678555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8074e-e559-4954-aff5-19a98dab521b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:54.678970 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8074e-e559-4954-aff5-19a98dab521b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:54.680047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8074e-e559-4954-aff5-19a98dab521b None admin] RBAC: Adding query filter params (role.id=0a6e69b6ee9e43659f71a5ba3260f5c9, user.id=b1c9ebb579ed4f2983b1ef77c59be8f5, scope.system=all) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:54.680499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8074e-e559-4954-aff5-19a98dab521b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:54.682519 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8074e-e559-4954-aff5-19a98dab521b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:54.688780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 165/309] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Jul 21 08:55:54 2022] GET /identity/v3/role_assignments?user.id=b1c9ebb579ed4f2983b1ef77c59be8f5&scope.system=all&role.id=0a6e69b6ee9e43659f71a5ba3260f5c9 => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:54.690133 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39ac63-a7fb-470a-84de-253ec6422e89 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:54.690434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39ac63-a7fb-470a-84de-253ec6422e89 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:54.690713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39ac63-a7fb-470a-84de-253ec6422e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:54.721134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20a65a-c707-42e9-801b-facb96ded352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:54.722470 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20a65a-c707-42e9-801b-facb96ded352 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:54.722581 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ba39ac63-a7fb-470a-84de-253ec6422e89 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:54.723053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20a65a-c707-42e9-801b-facb96ded352 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:54.723510 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20a65a-c707-42e9-801b-facb96ded352 None admin] PATH_INFO: `/v3/users/805ff5eabce744d2bc6b22ca4a6b3743` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:54.724284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ba39ac63-a7fb-470a-84de-253ec6422e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:54.725314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20a65a-c707-42e9-801b-facb96ded352 None admin] RBAC: Authorizing `identity:get_user(user_id=805ff5eabce744d2bc6b22ca4a6b3743)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:54.727442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20a65a-c707-42e9-801b-facb96ded352 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:54.730842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 145/310] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:54 2022] GET /identity/v3/users/805ff5eabce744d2bc6b22ca4a6b3743 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:54.741189 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20a65a-c707-42e9-801b-facb96ded352 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:54.834904 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 166/311] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:54.845987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:54.857792 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39ac63-a7fb-470a-84de-253ec6422e89 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:54.891786 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20a65a-c707-42e9-801b-facb96ded352 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:54.937351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc3c15-b130-42ce-b420-4d577233ce7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:54.938437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc3c15-b130-42ce-b420-4d577233ce7a None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:54.938931 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc3c15-b130-42ce-b420-4d577233ce7a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:54.939308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc3c15-b130-42ce-b420-4d577233ce7a None admin] PATH_INFO: `/v3/system/users/805ff5eabce744d2bc6b22ca4a6b3743/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:54.941052 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc3c15-b130-42ce-b420-4d577233ce7a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=805ff5eabce744d2bc6b22ca4a6b3743, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:54.942704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc3c15-b130-42ce-b420-4d577233ce7a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:54.955613 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 146/312] 10.222.0.13 () {58 vars in 1334 bytes} [Thu Jul 21 08:55:54 2022] PUT /identity/v3/system/users/805ff5eabce744d2bc6b22ca4a6b3743/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:55:54.981466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39ac63-a7fb-470a-84de-253ec6422e89 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:54.991145 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:55.023072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff166aa-0014-41b2-b7e4-130dec3a3545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:55.023963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff166aa-0014-41b2-b7e4-130dec3a3545 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:55.024257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff166aa-0014-41b2-b7e4-130dec3a3545 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:55.024535 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff166aa-0014-41b2-b7e4-130dec3a3545 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:55.037465 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff166aa-0014-41b2-b7e4-130dec3a3545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-8ff166aa-0014-41b2-b7e4-130dec3a3545 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:55.046822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:55.047770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:55.048576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:55.048576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:55.048576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:55.048576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:55.048576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:55.048576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:55.048576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:55.048576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:55.048576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:55.048576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:55.048576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:55.048576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:55.048576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 167/313] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:55.054834 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc3c15-b130-42ce-b420-4d577233ce7a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:55.177777 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc3c15-b130-42ce-b420-4d577233ce7a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:55.210726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:55.216603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd5f9e-79b7-49db-b974-e4cd6346464a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:55.217285 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd5f9e-79b7-49db-b974-e4cd6346464a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:55.217567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd5f9e-79b7-49db-b974-e4cd6346464a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:55.217840 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd5f9e-79b7-49db-b974-e4cd6346464a None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:55.218600 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd5f9e-79b7-49db-b974-e4cd6346464a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:55.218961 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd5f9e-79b7-49db-b974-e4cd6346464a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:55.220722 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd5f9e-79b7-49db-b974-e4cd6346464a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:55.232174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 147/314] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:55.238023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff166aa-0014-41b2-b7e4-130dec3a3545 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:55.349216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff166aa-0014-41b2-b7e4-130dec3a3545 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:55.385733 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77e36e-fb9e-414d-9059-cdbcc0eb5b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:55.386662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e36e-fb9e-414d-9059-cdbcc0eb5b69 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:55.386945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e36e-fb9e-414d-9059-cdbcc0eb5b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:55.387223 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77e36e-fb9e-414d-9059-cdbcc0eb5b69 None admin] PATH_INFO: `/v3/users/de475cb8d1634c3b93391c945297f483` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:55.388501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77e36e-fb9e-414d-9059-cdbcc0eb5b69 None admin] RBAC: Authorizing `identity:get_user(user_id=de475cb8d1634c3b93391c945297f483)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:55.389963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77e36e-fb9e-414d-9059-cdbcc0eb5b69 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:55.391866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 168/315] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:55 2022] GET /identity/v3/users/de475cb8d1634c3b93391c945297f483 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:55.397550 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd5f9e-79b7-49db-b974-e4cd6346464a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:55.523957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd5f9e-79b7-49db-b974-e4cd6346464a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:55.567186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf4c54-4b3c-4bf8-af16-8abcd8a32692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:55.567971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf4c54-4b3c-4bf8-af16-8abcd8a32692 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:55.568261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf4c54-4b3c-4bf8-af16-8abcd8a32692 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:55.568714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf4c54-4b3c-4bf8-af16-8abcd8a32692 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:55.659972 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf4c54-4b3c-4bf8-af16-8abcd8a32692 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:55.660753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:55.662353 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf4c54-4b3c-4bf8-af16-8abcd8a32692 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:55.799892 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 148/316] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:55:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 08:55:55.805496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77e36e-fb9e-414d-9059-cdbcc0eb5b69 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:55.843194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:55.914322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77e36e-fb9e-414d-9059-cdbcc0eb5b69 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:55.952405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79abd835-996d-4bae-a3d9-b7af14883dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:55.953300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abd835-996d-4bae-a3d9-b7af14883dee None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:55.953581 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abd835-996d-4bae-a3d9-b7af14883dee None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:55.953857 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abd835-996d-4bae-a3d9-b7af14883dee None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:55.955228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79abd835-996d-4bae-a3d9-b7af14883dee None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:55.956770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79abd835-996d-4bae-a3d9-b7af14883dee None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:55.958917 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 169/317] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:55:55 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:55.965051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf4c54-4b3c-4bf8-af16-8abcd8a32692 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:56.078623 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:56.085958 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf4c54-4b3c-4bf8-af16-8abcd8a32692 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:56.126294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c2339-e904-4d8a-a88f-daed26aefe61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:56.127022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c2339-e904-4d8a-a88f-daed26aefe61 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:56.127313 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c2339-e904-4d8a-a88f-daed26aefe61 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:56.127591 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c2339-e904-4d8a-a88f-daed26aefe61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:56.128487 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c2339-e904-4d8a-a88f-daed26aefe61 None admin] RBAC: Adding query filter params (role.id=0a6e69b6ee9e43659f71a5ba3260f5c9, scope.project.id=551f963b02f4411395171476c9cc2d68, user.id=de475cb8d1634c3b93391c945297f483) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:56.128804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c2339-e904-4d8a-a88f-daed26aefe61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:56.130318 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c2339-e904-4d8a-a88f-daed26aefe61 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:56.139703 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 149/318] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:55:55 2022] GET /identity/v3/role_assignments?user.id=de475cb8d1634c3b93391c945297f483&scope.project.id=551f963b02f4411395171476c9cc2d68&role.id=0a6e69b6ee9e43659f71a5ba3260f5c9 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:56.690000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad14b31-bde3-4c3c-875c-8615b7251c75 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:56.690528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad14b31-bde3-4c3c-875c-8615b7251c75 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:56.690528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad14b31-bde3-4c3c-875c-8615b7251c75 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:56.691992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 170/319] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:56.698114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c1d49-06f8-4fee-92f2-448272776fa1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:56.698503 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c1d49-06f8-4fee-92f2-448272776fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:56.698808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c1d49-06f8-4fee-92f2-448272776fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:56.721893 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3d5c1d49-06f8-4fee-92f2-448272776fa1 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:56.723115 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3d5c1d49-06f8-4fee-92f2-448272776fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:56.806743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 150/320] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:56.830169 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad14b31-bde3-4c3c-875c-8615b7251c75 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:56.954143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad14b31-bde3-4c3c-875c-8615b7251c75 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:56.954325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0722dc-a789-4416-882b-fb27935ccbca None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:56.954405 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0722dc-a789-4416-882b-fb27935ccbca None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:56.954825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0722dc-a789-4416-882b-fb27935ccbca None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:56.955751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:56.956667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 151/321] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:55:56.961875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68117740-6cbe-4b19-b306-6dcd37847be9 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:56.962527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68117740-6cbe-4b19-b306-6dcd37847be9 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:56.963436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68117740-6cbe-4b19-b306-6dcd37847be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:56.983053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-68117740-6cbe-4b19-b306-6dcd37847be9 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:56.985069 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-68117740-6cbe-4b19-b306-6dcd37847be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:56.992910 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32edaa66-0bac-47ed-8034-c2086ea1276b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:56.994154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32edaa66-0bac-47ed-8034-c2086ea1276b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:56.994448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32edaa66-0bac-47ed-8034-c2086ea1276b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:56.994729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32edaa66-0bac-47ed-8034-c2086ea1276b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:56.996162 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32edaa66-0bac-47ed-8034-c2086ea1276b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:56.997631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32edaa66-0bac-47ed-8034-c2086ea1276b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:56.999479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 171/322] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:55:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:57.004977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32edaa66-0bac-47ed-8034-c2086ea1276b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:57.060125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 152/323] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:55:57.082710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68117740-6cbe-4b19-b306-6dcd37847be9 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:57.101941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:57.118672 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32edaa66-0bac-47ed-8034-c2086ea1276b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:57.155821 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822be0a9-0e4f-4588-bb3a-d1e9814e3a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:57.156530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822be0a9-0e4f-4588-bb3a-d1e9814e3a1f None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:57.156822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822be0a9-0e4f-4588-bb3a-d1e9814e3a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:57.157099 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822be0a9-0e4f-4588-bb3a-d1e9814e3a1f None admin] PATH_INFO: `/v3/groups` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:57.157806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822be0a9-0e4f-4588-bb3a-d1e9814e3a1f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:57.159283 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822be0a9-0e4f-4588-bb3a-d1e9814e3a1f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:57.176740 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 172/324] 10.222.0.13 () {60 vars in 1134 bytes} [Thu Jul 21 08:55:57 2022] POST /identity/v3/groups => generated 226 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:55:57.185620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d0408-7791-4ddb-989e-8490126989b5 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:57.185902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d0408-7791-4ddb-989e-8490126989b5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:57.186180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d0408-7791-4ddb-989e-8490126989b5 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:57.187764 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 173/325] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:57.194415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404dd97-826a-4f18-9b5d-3617766fae75 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:57.194750 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404dd97-826a-4f18-9b5d-3617766fae75 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:57.195503 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404dd97-826a-4f18-9b5d-3617766fae75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:57.198128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68117740-6cbe-4b19-b306-6dcd37847be9 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:57.215375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c404dd97-826a-4f18-9b5d-3617766fae75 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:57.216576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c404dd97-826a-4f18-9b5d-3617766fae75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:57.242541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c45fd-70bf-4270-b57b-23e4fb3037a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:57.243664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c45fd-70bf-4270-b57b-23e4fb3037a2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:57.244196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c45fd-70bf-4270-b57b-23e4fb3037a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:57.244624 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:57.244764 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c45fd-70bf-4270-b57b-23e4fb3037a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:57.257767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c45fd-70bf-4270-b57b-23e4fb3037a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-700c45fd-70bf-4270-b57b-23e4fb3037a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:57.266489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:57.267505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:57.268618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:57.268618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:57.268618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:57.268618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:57.268618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:57.268618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:57.268618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:57.268618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:57.268618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:57.268618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:57.268618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:55:57.268618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:55:57.268618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 153/326] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:55:57.273771 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c45fd-70bf-4270-b57b-23e4fb3037a2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:57.299430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 174/327] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:55:57.324577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404dd97-826a-4f18-9b5d-3617766fae75 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:57.377111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:57.410159 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c45fd-70bf-4270-b57b-23e4fb3037a2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:57.448166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404dd97-826a-4f18-9b5d-3617766fae75 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:57.463143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351ac54-4ebd-491d-bfcf-5987da3f052c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:57.463846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351ac54-4ebd-491d-bfcf-5987da3f052c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:57.464204 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351ac54-4ebd-491d-bfcf-5987da3f052c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:57.464495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351ac54-4ebd-491d-bfcf-5987da3f052c None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:57.465271 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351ac54-4ebd-491d-bfcf-5987da3f052c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:57.465707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351ac54-4ebd-491d-bfcf-5987da3f052c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:57.467093 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351ac54-4ebd-491d-bfcf-5987da3f052c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:57.481804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 154/328] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:57.487525 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351ac54-4ebd-491d-bfcf-5987da3f052c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:57.498004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caeae6e-7bb4-4d7c-9e8f-f0a4808b5535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:57.498346 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caeae6e-7bb4-4d7c-9e8f-f0a4808b5535 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:57.498622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caeae6e-7bb4-4d7c-9e8f-f0a4808b5535 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:57.498887 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caeae6e-7bb4-4d7c-9e8f-f0a4808b5535 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:57.515690 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caeae6e-7bb4-4d7c-9e8f-f0a4808b5535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-1caeae6e-7bb4-4d7c-9e8f-f0a4808b5535 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:55:57.523520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:55:57.524499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:55:57.525300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:55:57.525300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:55:57.525300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:55:57.525300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:55:57.525300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:55:57.525300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:55:57.525300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:55:57.525300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:55:57.525300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:55:57.525300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:55:57.525300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:55:57.525300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:55:57.525300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 175/329] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:55:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:55:57.531621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caeae6e-7bb4-4d7c-9e8f-f0a4808b5535 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:57.560081 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:57.622465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351ac54-4ebd-491d-bfcf-5987da3f052c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:57.676849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec901e-8db6-4ea3-8508-96848a9e2267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:57.677737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec901e-8db6-4ea3-8508-96848a9e2267 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:57.678022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec901e-8db6-4ea3-8508-96848a9e2267 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:57.678198 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caeae6e-7bb4-4d7c-9e8f-f0a4808b5535 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:57.678320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec901e-8db6-4ea3-8508-96848a9e2267 None admin] PATH_INFO: `/v3/users/b1c9ebb579ed4f2983b1ef77c59be8f5` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:57.679533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec901e-8db6-4ea3-8508-96848a9e2267 None admin] RBAC: Authorizing `identity:get_user(user_id=b1c9ebb579ed4f2983b1ef77c59be8f5)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:57.681067 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec901e-8db6-4ea3-8508-96848a9e2267 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:57.683321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 155/330] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:57 2022] GET /identity/v3/users/b1c9ebb579ed4f2983b1ef77c59be8f5 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:57.692938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec901e-8db6-4ea3-8508-96848a9e2267 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:57.711794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:57.732697 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de6b4f-e767-46ef-b1c0-b71ab86b8b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:57.734776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de6b4f-e767-46ef-b1c0-b71ab86b8b80 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:57.735248 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de6b4f-e767-46ef-b1c0-b71ab86b8b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:57.735763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de6b4f-e767-46ef-b1c0-b71ab86b8b80 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:57.737372 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de6b4f-e767-46ef-b1c0-b71ab86b8b80 None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:57.737974 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de6b4f-e767-46ef-b1c0-b71ab86b8b80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:57.739900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de6b4f-e767-46ef-b1c0-b71ab86b8b80 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:57.752228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 176/331] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:55:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:57.759376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de6b4f-e767-46ef-b1c0-b71ab86b8b80 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:57.848200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec901e-8db6-4ea3-8508-96848a9e2267 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:57.890410 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:57.897980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76fce1-f7d2-43eb-b380-1ea4689a72c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:57.898938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76fce1-f7d2-43eb-b380-1ea4689a72c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:57.899291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76fce1-f7d2-43eb-b380-1ea4689a72c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:57.899627 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76fce1-f7d2-43eb-b380-1ea4689a72c2 None admin] PATH_INFO: `/v3/system/users/b1c9ebb579ed4f2983b1ef77c59be8f5/roles/0a6e69b6ee9e43659f71a5ba3260f5c9` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:57.901260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76fce1-f7d2-43eb-b380-1ea4689a72c2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b1c9ebb579ed4f2983b1ef77c59be8f5, role_id=0a6e69b6ee9e43659f71a5ba3260f5c9)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:57.903594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76fce1-f7d2-43eb-b380-1ea4689a72c2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:57.914162 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 156/332] 10.222.0.13 () {58 vars in 1334 bytes} [Thu Jul 21 08:55:57 2022] PUT /identity/v3/system/users/b1c9ebb579ed4f2983b1ef77c59be8f5/roles/0a6e69b6ee9e43659f71a5ba3260f5c9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:55:57.923578 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de6b4f-e767-46ef-b1c0-b71ab86b8b80 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:57.971298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8f6e4-fa6c-47c6-8544-dffb9da9614e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:57.972227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8f6e4-fa6c-47c6-8544-dffb9da9614e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:57.972515 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8f6e4-fa6c-47c6-8544-dffb9da9614e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:57.972793 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8f6e4-fa6c-47c6-8544-dffb9da9614e None admin] PATH_INFO: `/v3/users/805ff5eabce744d2bc6b22ca4a6b3743` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:57.974043 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8f6e4-fa6c-47c6-8544-dffb9da9614e None admin] RBAC: Authorizing `identity:get_user(user_id=805ff5eabce744d2bc6b22ca4a6b3743)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:57.975628 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8f6e4-fa6c-47c6-8544-dffb9da9614e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:57.977876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 177/333] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:55:57 2022] GET /identity/v3/users/805ff5eabce744d2bc6b22ca4a6b3743 => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:55:57.985078 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76fce1-f7d2-43eb-b380-1ea4689a72c2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:55:58.075861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:55:58.113812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76fce1-f7d2-43eb-b380-1ea4689a72c2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:55:58.155100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea9937-a13a-48a7-9db7-9899ba28ae9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:55:58.156194 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea9937-a13a-48a7-9db7-9899ba28ae9c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:58.156720 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea9937-a13a-48a7-9db7-9899ba28ae9c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:58.157214 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea9937-a13a-48a7-9db7-9899ba28ae9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:58.158065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea9937-a13a-48a7-9db7-9899ba28ae9c None admin] RBAC: Adding query filter params (role.id=3d7c80cf23864f58bf21f893ba7691d6, user.id=805ff5eabce744d2bc6b22ca4a6b3743, scope.system=all) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:55:58.158382 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea9937-a13a-48a7-9db7-9899ba28ae9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:55:58.159869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea9937-a13a-48a7-9db7-9899ba28ae9c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:55:58.174379 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 157/334] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Jul 21 08:55:57 2022] GET /identity/v3/role_assignments?user.id=805ff5eabce744d2bc6b22ca4a6b3743&scope.system=all&role.id=3d7c80cf23864f58bf21f893ba7691d6 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:55:59.865696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd1199-5ab9-4808-9abe-392d4dbff088 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:59.866239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd1199-5ab9-4808-9abe-392d4dbff088 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:59.866239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd1199-5ab9-4808-9abe-392d4dbff088 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:59.867990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 178/335] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:55:59.874723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7be04-f9a9-4ff3-815f-d64e4d1f2dcf None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:59.875231 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7be04-f9a9-4ff3-815f-d64e4d1f2dcf None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:59.875320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7be04-f9a9-4ff3-815f-d64e4d1f2dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:59.901700 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f7b7be04-f9a9-4ff3-815f-d64e4d1f2dcf None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:59.903173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f7b7be04-f9a9-4ff3-815f-d64e4d1f2dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:55:59.934876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da48410-fe00-4505-9f46-0c03674d7771 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:59.935210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da48410-fe00-4505-9f46-0c03674d7771 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:59.935435 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da48410-fe00-4505-9f46-0c03674d7771 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:59.938345 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 179/336] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:55:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:55:59.943541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bff4f-03f3-4cbe-ada7-2b9ca4a10e5a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:55:59.943820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bff4f-03f3-4cbe-ada7-2b9ca4a10e5a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:55:59.944130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bff4f-03f3-4cbe-ada7-2b9ca4a10e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:55:59.966072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-447bff4f-03f3-4cbe-ada7-2b9ca4a10e5a None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:55:59.967241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-447bff4f-03f3-4cbe-ada7-2b9ca4a10e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:00.004758 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 158/337] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:00.054725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7be04-f9a9-4ff3-815f-d64e4d1f2dcf None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:00.072038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 180/338] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:56:00.148574 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bff4f-03f3-4cbe-ada7-2b9ca4a10e5a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:00.164043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:00.252161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7be04-f9a9-4ff3-815f-d64e4d1f2dcf None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:00.368127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b70770-36a0-4e60-a31b-682b9b01ebfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:00.369625 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b70770-36a0-4e60-a31b-682b9b01ebfd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:00.370138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b70770-36a0-4e60-a31b-682b9b01ebfd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:00.370637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b70770-36a0-4e60-a31b-682b9b01ebfd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:00.381171 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:00.391136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b70770-36a0-4e60-a31b-682b9b01ebfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-18b70770-36a0-4e60-a31b-682b9b01ebfd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:00.399070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:00.400861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:00.402963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:00.402963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:00.402963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:00.402963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:00.402963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:00.402963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:00.402963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:00.402963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:00.402963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:00.402963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:00.402963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:56:00.402963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:00.408814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 159/339] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:00.429401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bff4f-03f3-4cbe-ada7-2b9ca4a10e5a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:00.454284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b70770-36a0-4e60-a31b-682b9b01ebfd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:00.493437 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1d0ca-a075-4419-8c47-145dbf1688ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:00.494774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1d0ca-a075-4419-8c47-145dbf1688ce None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:00.495489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1d0ca-a075-4419-8c47-145dbf1688ce None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:00.496170 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec1d0ca-a075-4419-8c47-145dbf1688ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:00.509502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec1d0ca-a075-4419-8c47-145dbf1688ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-3ec1d0ca-a075-4419-8c47-145dbf1688ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:00.519316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:00.520311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:00.521087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:00.521087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:00.521087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:00.521087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:00.521087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:00.521087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:00.521087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:00.521087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:00.521087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:00.521087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:00.521087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:56:00.521087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:00.524111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 181/340] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:56:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:00.530906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1d0ca-a075-4419-8c47-145dbf1688ce None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:00.687800 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec1d0ca-a075-4419-8c47-145dbf1688ce None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:00.718273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b70770-36a0-4e60-a31b-682b9b01ebfd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:00.741825 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01caba3-7540-4554-9199-0af42c1c368c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:00.742916 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01caba3-7540-4554-9199-0af42c1c368c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:00.743633 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01caba3-7540-4554-9199-0af42c1c368c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:00.744254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01caba3-7540-4554-9199-0af42c1c368c None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:00.745345 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01caba3-7540-4554-9199-0af42c1c368c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:00.746543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01caba3-7540-4554-9199-0af42c1c368c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:00.749028 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:00.750109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01caba3-7540-4554-9199-0af42c1c368c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:00.764429 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 182/341] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:00.770148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01caba3-7540-4554-9199-0af42c1c368c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:00.776688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:00.792972 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f25efa-a0a2-4131-a6fd-845d5b9b1069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:00.793811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f25efa-a0a2-4131-a6fd-845d5b9b1069 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:00.794514 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f25efa-a0a2-4131-a6fd-845d5b9b1069 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:00.794514 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f25efa-a0a2-4131-a6fd-845d5b9b1069 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:00.795123 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f25efa-a0a2-4131-a6fd-845d5b9b1069 None admin] RBAC: Adding query filter params (name=member) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:00.795426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f25efa-a0a2-4131-a6fd-845d5b9b1069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:00.796917 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f25efa-a0a2-4131-a6fd-845d5b9b1069 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:00.814955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 160/342] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:56:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:00.816403 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0e5cb-37d4-4b38-8a55-172115c4b335 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:00.816697 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0e5cb-37d4-4b38-8a55-172115c4b335 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:00.816971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0e5cb-37d4-4b38-8a55-172115c4b335 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:00.818238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 161/343] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:00.824507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61233031-0b47-4695-b93e-40bed4e2a2e7 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:00.824809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61233031-0b47-4695-b93e-40bed4e2a2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:00.825091 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61233031-0b47-4695-b93e-40bed4e2a2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:00.869012 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-61233031-0b47-4695-b93e-40bed4e2a2e7 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:00.870480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-61233031-0b47-4695-b93e-40bed4e2a2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:00.908034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01caba3-7540-4554-9199-0af42c1c368c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:00.950006 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d819c03-f4a4-4ca5-8dd2-b9df3583b8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:00.951024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d819c03-f4a4-4ca5-8dd2-b9df3583b8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:00.951548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d819c03-f4a4-4ca5-8dd2-b9df3583b8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:00.952061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d819c03-f4a4-4ca5-8dd2-b9df3583b8b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:01.002806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 162/344] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:01.003821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61233031-0b47-4695-b93e-40bed4e2a2e7 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:01.030439 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d819c03-f4a4-4ca5-8dd2-b9df3583b8b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:01.033384 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d819c03-f4a4-4ca5-8dd2-b9df3583b8b7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:01.037302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:01.164378 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 183/345] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 08:56:01.165977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d819c03-f4a4-4ca5-8dd2-b9df3583b8b7 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:01.191167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61233031-0b47-4695-b93e-40bed4e2a2e7 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:01.216698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:01.260664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f1ef6-9881-4eac-aa1f-e71a8885aeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:01.262009 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f1ef6-9881-4eac-aa1f-e71a8885aeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:01.262469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f1ef6-9881-4eac-aa1f-e71a8885aeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:01.262801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f1ef6-9881-4eac-aa1f-e71a8885aeb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:01.288576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d819c03-f4a4-4ca5-8dd2-b9df3583b8b7 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:01.303652 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:01.334654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af1487-cd2e-4c0f-bee3-336edc8586fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:01.335957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af1487-cd2e-4c0f-bee3-336edc8586fc None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:01.336448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af1487-cd2e-4c0f-bee3-336edc8586fc None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:01.336914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af1487-cd2e-4c0f-bee3-336edc8586fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:01.350866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af1487-cd2e-4c0f-bee3-336edc8586fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-c2af1487-cd2e-4c0f-bee3-336edc8586fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:01.361284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:01.362313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:01.363100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:01.363100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:01.363100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:01.363100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:01.363100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:01.363100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:01.363100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:01.363100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:01.363100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:01.363100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:01.363100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:56:01.363100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:01.363693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f1ef6-9881-4eac-aa1f-e71a8885aeb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:01.364418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f1ef6-9881-4eac-aa1f-e71a8885aeb4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:01.366229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 184/346] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:01.367554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af1487-cd2e-4c0f-bee3-336edc8586fc None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:01.485724 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af1487-cd2e-4c0f-bee3-336edc8586fc None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:01.523071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 163/347] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:01.523071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f1ef6-9881-4eac-aa1f-e71a8885aeb4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:01.534730 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05ab90-dfe3-444a-a823-7fe689a14f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:01.536092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05ab90-dfe3-444a-a823-7fe689a14f6d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:01.536708 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05ab90-dfe3-444a-a823-7fe689a14f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:01.541888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05ab90-dfe3-444a-a823-7fe689a14f6d None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:01.542416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:01.544666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05ab90-dfe3-444a-a823-7fe689a14f6d None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:01.546783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05ab90-dfe3-444a-a823-7fe689a14f6d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:01.549461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 185/348] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:56:01 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:01.550564 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05ab90-dfe3-444a-a823-7fe689a14f6d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:01.632380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:01.737621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05ab90-dfe3-444a-a823-7fe689a14f6d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:01.794250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc6e3f-a443-410f-9d3b-2f276cc1f45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:01.795597 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc6e3f-a443-410f-9d3b-2f276cc1f45a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:01.795886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc6e3f-a443-410f-9d3b-2f276cc1f45a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:01.796226 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc6e3f-a443-410f-9d3b-2f276cc1f45a None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:01.797534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc6e3f-a443-410f-9d3b-2f276cc1f45a None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:01.799050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc6e3f-a443-410f-9d3b-2f276cc1f45a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:01.802768 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 186/349] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:56:01 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:01.804257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc6e3f-a443-410f-9d3b-2f276cc1f45a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:01.827032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f1ef6-9881-4eac-aa1f-e71a8885aeb4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:01.910821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ac5a4-617d-4be5-b46e-a57fe9603caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:01.911505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ac5a4-617d-4be5-b46e-a57fe9603caf None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:01.911790 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ac5a4-617d-4be5-b46e-a57fe9603caf None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:01.912090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ac5a4-617d-4be5-b46e-a57fe9603caf None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:01.912854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ac5a4-617d-4be5-b46e-a57fe9603caf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:01.913156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ac5a4-617d-4be5-b46e-a57fe9603caf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:01.914545 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ac5a4-617d-4be5-b46e-a57fe9603caf None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:01.927203 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 164/350] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:56:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:01.927560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ac5a4-617d-4be5-b46e-a57fe9603caf None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:01.975623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:01.983337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc6e3f-a443-410f-9d3b-2f276cc1f45a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:02.029916 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16138b9c-5623-4ebe-a5f3-b778338a2651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:02.031156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16138b9c-5623-4ebe-a5f3-b778338a2651 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:02.031666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16138b9c-5623-4ebe-a5f3-b778338a2651 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:02.032187 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16138b9c-5623-4ebe-a5f3-b778338a2651 None admin] PATH_INFO: `/v3/groups/138bdc814b3c4eb9885c7cb5da450386` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:02.041323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16138b9c-5623-4ebe-a5f3-b778338a2651 None admin] RBAC: Authorizing `identity:get_group(group_id=138bdc814b3c4eb9885c7cb5da450386)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:02.044991 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16138b9c-5623-4ebe-a5f3-b778338a2651 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:02.046932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 187/351] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jul 21 08:56:01 2022] GET /identity/v3/groups/138bdc814b3c4eb9885c7cb5da450386 => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:02.048994 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68116-7050-4108-89c6-ea4bf2290481 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:02.048994 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68116-7050-4108-89c6-ea4bf2290481 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:02.048994 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68116-7050-4108-89c6-ea4bf2290481 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:02.050011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:02.053111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 188/352] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:02.053111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f68116-7050-4108-89c6-ea4bf2290481 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:02.091353 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ac5a4-617d-4be5-b46e-a57fe9603caf None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:02.139121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011c325-63f8-4d99-aeea-dfff2d2ce6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:02.141029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011c325-63f8-4d99-aeea-dfff2d2ce6af None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:02.141029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011c325-63f8-4d99-aeea-dfff2d2ce6af None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:02.141029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011c325-63f8-4d99-aeea-dfff2d2ce6af None admin] PATH_INFO: `/v3/groups/138bdc814b3c4eb9885c7cb5da450386` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:02.142818 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011c325-63f8-4d99-aeea-dfff2d2ce6af None admin] RBAC: Authorizing `identity:get_group(group_id=138bdc814b3c4eb9885c7cb5da450386)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:02.143457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011c325-63f8-4d99-aeea-dfff2d2ce6af None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:02.146077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 165/353] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jul 21 08:56:01 2022] GET /identity/v3/groups/138bdc814b3c4eb9885c7cb5da450386 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:02.147315 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900fa7b-dbc0-44e0-8653-56713e496e48 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:02.147600 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900fa7b-dbc0-44e0-8653-56713e496e48 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:02.147875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900fa7b-dbc0-44e0-8653-56713e496e48 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:02.154922 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 166/354] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:02.156419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aaa71-86b9-46d7-9c62-8cf1a7ab20ba None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:02.156928 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aaa71-86b9-46d7-9c62-8cf1a7ab20ba None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:02.157402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aaa71-86b9-46d7-9c62-8cf1a7ab20ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:02.179266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ae7aaa71-86b9-46d7-9c62-8cf1a7ab20ba None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:02.180471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ae7aaa71-86b9-46d7-9c62-8cf1a7ab20ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:02.199757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f68116-7050-4108-89c6-ea4bf2290481 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:02.249493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dd30c-96c6-4a62-b28d-a71d542d1678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:02.250611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dd30c-96c6-4a62-b28d-a71d542d1678 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:02.251068 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dd30c-96c6-4a62-b28d-a71d542d1678 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:02.251959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dd30c-96c6-4a62-b28d-a71d542d1678 None admin] PATH_INFO: `/v3/users/b1c9ebb579ed4f2983b1ef77c59be8f5` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:02.253301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9dd30c-96c6-4a62-b28d-a71d542d1678 None admin] RBAC: Authorizing `identity:get_user(user_id=b1c9ebb579ed4f2983b1ef77c59be8f5)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:02.254783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9dd30c-96c6-4a62-b28d-a71d542d1678 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:02.256723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 189/355] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:56:02 2022] GET /identity/v3/users/b1c9ebb579ed4f2983b1ef77c59be8f5 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:02.257366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dd30c-96c6-4a62-b28d-a71d542d1678 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:02.275051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 167/356] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:02.276214 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7aaa71-86b9-46d7-9c62-8cf1a7ab20ba None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:02.294462 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:02.369257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:02.396393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dd30c-96c6-4a62-b28d-a71d542d1678 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:02.404920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7aaa71-86b9-46d7-9c62-8cf1a7ab20ba None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:02.437553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5dd57f-8030-4e3b-9ff7-8cb32b71f8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:02.438214 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dd57f-8030-4e3b-9ff7-8cb32b71f8fd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:02.438501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dd57f-8030-4e3b-9ff7-8cb32b71f8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:02.438779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dd57f-8030-4e3b-9ff7-8cb32b71f8fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:02.439618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5dd57f-8030-4e3b-9ff7-8cb32b71f8fd None admin] RBAC: Adding query filter params (group.id=138bdc814b3c4eb9885c7cb5da450386, role.id=5a989bb8c8a1465ca76622ffb504dee5, scope.project.id=e1099d961ac5441c973dfbd04812f431) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:02.439943 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5dd57f-8030-4e3b-9ff7-8cb32b71f8fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:02.441406 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5dd57f-8030-4e3b-9ff7-8cb32b71f8fd None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:02.445513 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888cb8d-7399-4dd7-aa3f-f4b9bf259050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:02.446164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888cb8d-7399-4dd7-aa3f-f4b9bf259050 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:02.446442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888cb8d-7399-4dd7-aa3f-f4b9bf259050 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:02.446713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888cb8d-7399-4dd7-aa3f-f4b9bf259050 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:02.447516 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 190/357] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jul 21 08:56:02 2022] GET /identity/v3/role_assignments?group.id=138bdc814b3c4eb9885c7cb5da450386&scope.project.id=e1099d961ac5441c973dfbd04812f431&role.id=5a989bb8c8a1465ca76622ffb504dee5 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:02.447624 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888cb8d-7399-4dd7-aa3f-f4b9bf259050 None admin] RBAC: Adding query filter params (group.id=138bdc814b3c4eb9885c7cb5da450386, role.id=3d7c80cf23864f58bf21f893ba7691d6, scope.project.id=e1099d961ac5441c973dfbd04812f431) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:02.447847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888cb8d-7399-4dd7-aa3f-f4b9bf259050 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:02.449949 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888cb8d-7399-4dd7-aa3f-f4b9bf259050 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:02.452953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251ff7d-e407-4f67-8b48-672c435143c5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:02.453242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251ff7d-e407-4f67-8b48-672c435143c5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:02.453524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251ff7d-e407-4f67-8b48-672c435143c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:02.461905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 168/358] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jul 21 08:56:02 2022] GET /identity/v3/role_assignments?group.id=138bdc814b3c4eb9885c7cb5da450386&scope.project.id=e1099d961ac5441c973dfbd04812f431&role.id=3d7c80cf23864f58bf21f893ba7691d6 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:02.462601 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888cb8d-7399-4dd7-aa3f-f4b9bf259050 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:02.481470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b251ff7d-e407-4f67-8b48-672c435143c5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:02.483196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b251ff7d-e407-4f67-8b48-672c435143c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:02.580071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 191/359] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:02.582524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251ff7d-e407-4f67-8b48-672c435143c5 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:02.616710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:02.700040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888cb8d-7399-4dd7-aa3f-f4b9bf259050 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:02.747433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251ff7d-e407-4f67-8b48-672c435143c5 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:02.747746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34a29f-4562-4086-8011-45d55b0963d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:02.748468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34a29f-4562-4086-8011-45d55b0963d4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:02.748754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34a29f-4562-4086-8011-45d55b0963d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:02.749034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34a29f-4562-4086-8011-45d55b0963d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:02.749890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34a29f-4562-4086-8011-45d55b0963d4 None admin] RBAC: Adding query filter params (role.id=0a6e69b6ee9e43659f71a5ba3260f5c9, user.id=b1c9ebb579ed4f2983b1ef77c59be8f5, scope.system=all) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:02.750206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34a29f-4562-4086-8011-45d55b0963d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:02.751715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34a29f-4562-4086-8011-45d55b0963d4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:02.755150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:02.770787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 169/360] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Jul 21 08:56:02 2022] GET /identity/v3/role_assignments?user.id=b1c9ebb579ed4f2983b1ef77c59be8f5&scope.system=all&role.id=0a6e69b6ee9e43659f71a5ba3260f5c9 => generated 510 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:02.772159 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09d700-c9a0-418f-ab70-0f2dce9aa09e None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:02.772448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09d700-c9a0-418f-ab70-0f2dce9aa09e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:02.772722 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09d700-c9a0-418f-ab70-0f2dce9aa09e None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:02.774001 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 170/361] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:02.774729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09d700-c9a0-418f-ab70-0f2dce9aa09e None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:02.822126 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23725e7f-da86-4336-9bce-9809d3edab63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:02.823004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23725e7f-da86-4336-9bce-9809d3edab63 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:02.823291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23725e7f-da86-4336-9bce-9809d3edab63 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:02.823568 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23725e7f-da86-4336-9bce-9809d3edab63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:02.843895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23725e7f-da86-4336-9bce-9809d3edab63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-23725e7f-da86-4336-9bce-9809d3edab63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:02.858553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:02.861154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:02.863061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:02.863061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:02.863061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:02.863061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:02.863061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:02.863061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:02.863061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:02.863061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:02.863061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:02.863061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:02.863061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:56:02.863061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:02.863061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 192/362] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:02.870323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701d9be-b661-4585-a16a-e72742a7511f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:02.870323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701d9be-b661-4585-a16a-e72742a7511f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:02.870323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701d9be-b661-4585-a16a-e72742a7511f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:02.896152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a701d9be-b661-4585-a16a-e72742a7511f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:02.897418 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a701d9be-b661-4585-a16a-e72742a7511f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:03.018337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 193/363] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:03.019137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701d9be-b661-4585-a16a-e72742a7511f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:03.021180 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09d700-c9a0-418f-ab70-0f2dce9aa09e None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:03.040272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:03.092725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89aec62-c7e6-4c87-808d-f7af7175f046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:03.093551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89aec62-c7e6-4c87-808d-f7af7175f046 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:03.093838 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89aec62-c7e6-4c87-808d-f7af7175f046 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:03.094173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89aec62-c7e6-4c87-808d-f7af7175f046 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:03.121768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89aec62-c7e6-4c87-808d-f7af7175f046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-e89aec62-c7e6-4c87-808d-f7af7175f046 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:03.134278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:03.135491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:03.136437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:03.136437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:03.136437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:03.136437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:03.136437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:03.136437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:03.136437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:03.136437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:03.136437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:03.136437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:03.136437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:56:03.136437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:03.137405 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 171/364] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:56:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:03.139785 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89aec62-c7e6-4c87-808d-f7af7175f046 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:03.179897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701d9be-b661-4585-a16a-e72742a7511f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:03.213850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:03.236705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c5249-d417-45f0-a966-897558dbdb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:03.237844 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c5249-d417-45f0-a966-897558dbdb2c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:03.238369 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c5249-d417-45f0-a966-897558dbdb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:03.238863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c5249-d417-45f0-a966-897558dbdb2c None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:03.239945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c5249-d417-45f0-a966-897558dbdb2c None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:03.240490 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c5249-d417-45f0-a966-897558dbdb2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:03.243041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c5249-d417-45f0-a966-897558dbdb2c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:03.257367 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 194/365] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:56:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:03.259166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c5249-d417-45f0-a966-897558dbdb2c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:03.282943 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89aec62-c7e6-4c87-808d-f7af7175f046 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:03.328600 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98386ea1-17ce-450a-85d5-f45112745188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:03.329549 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98386ea1-17ce-450a-85d5-f45112745188 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:03.329769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98386ea1-17ce-450a-85d5-f45112745188 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:03.330043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98386ea1-17ce-450a-85d5-f45112745188 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:03.343996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98386ea1-17ce-450a-85d5-f45112745188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-98386ea1-17ce-450a-85d5-f45112745188 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:03.352195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:03.353158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:03.353960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:03.353960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:03.353960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:03.353960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:03.353960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:03.353960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:03.353960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:03.353960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:03.353960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:03.353960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:03.353960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:03.353960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:03.353960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 172/366] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:56:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:03.354622 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98386ea1-17ce-450a-85d5-f45112745188 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:03.378156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:03.407388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c5249-d417-45f0-a966-897558dbdb2c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:03.446919 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426ec37-5709-4336-bf77-6bfdf0132dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:03.448010 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426ec37-5709-4336-bf77-6bfdf0132dcf None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:03.448503 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426ec37-5709-4336-bf77-6bfdf0132dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:03.448981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426ec37-5709-4336-bf77-6bfdf0132dcf None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:03.450304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426ec37-5709-4336-bf77-6bfdf0132dcf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:03.450831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426ec37-5709-4336-bf77-6bfdf0132dcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:03.452525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426ec37-5709-4336-bf77-6bfdf0132dcf None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:03.462083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 195/367] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:03.463554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426ec37-5709-4336-bf77-6bfdf0132dcf None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:03.474148 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98386ea1-17ce-450a-85d5-f45112745188 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:03.489995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:03.512769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4420f8a-96f0-4c24-8ef2-8c9e86e9cf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:03.513429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4420f8a-96f0-4c24-8ef2-8c9e86e9cf1b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:03.513761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4420f8a-96f0-4c24-8ef2-8c9e86e9cf1b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:03.514045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4420f8a-96f0-4c24-8ef2-8c9e86e9cf1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:03.579897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426ec37-5709-4336-bf77-6bfdf0132dcf None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:03.607094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4420f8a-96f0-4c24-8ef2-8c9e86e9cf1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:03.608829 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4420f8a-96f0-4c24-8ef2-8c9e86e9cf1b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:03.620001 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f8eac-8b34-44b5-9742-6d2ca3743f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:03.621641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f8eac-8b34-44b5-9742-6d2ca3743f68 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:03.622131 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f8eac-8b34-44b5-9742-6d2ca3743f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:03.622609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f8eac-8b34-44b5-9742-6d2ca3743f68 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:03.623664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f8eac-8b34-44b5-9742-6d2ca3743f68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:03.624208 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f8eac-8b34-44b5-9742-6d2ca3743f68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:03.625908 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f8eac-8b34-44b5-9742-6d2ca3743f68 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:03.638073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 196/368] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:03.639352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f8eac-8b34-44b5-9742-6d2ca3743f68 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:03.647397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:03.757022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 173/369] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:03.758875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4420f8a-96f0-4c24-8ef2-8c9e86e9cf1b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:03.780277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:03.787546 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f8eac-8b34-44b5-9742-6d2ca3743f68 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:03.840836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b738e6c-0ad4-4d69-b0e2-3d79f6f513ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:03.841732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b738e6c-0ad4-4d69-b0e2-3d79f6f513ea None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:03.842020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b738e6c-0ad4-4d69-b0e2-3d79f6f513ea None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:03.842296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b738e6c-0ad4-4d69-b0e2-3d79f6f513ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:03.920170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4420f8a-96f0-4c24-8ef2-8c9e86e9cf1b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:03.946144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:03.948982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b738e6c-0ad4-4d69-b0e2-3d79f6f513ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:03.951407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b738e6c-0ad4-4d69-b0e2-3d79f6f513ea None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:03.960944 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bba49-d771-4a02-a4ab-d8a2de6f3b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:03.961749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bba49-d771-4a02-a4ab-d8a2de6f3b45 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:03.962040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bba49-d771-4a02-a4ab-d8a2de6f3b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:03.962321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bba49-d771-4a02-a4ab-d8a2de6f3b45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:04.048222 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bba49-d771-4a02-a4ab-d8a2de6f3b45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:04.048837 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bba49-d771-4a02-a4ab-d8a2de6f3b45 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:04.066484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:04.114314 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 197/370] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:04.116806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b738e6c-0ad4-4d69-b0e2-3d79f6f513ea None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:04.183386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 174/371] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:04.185801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bba49-d771-4a02-a4ab-d8a2de6f3b45 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:04.230150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:04.237285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b738e6c-0ad4-4d69-b0e2-3d79f6f513ea None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:04.283344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1158c-f9c6-4747-aa5e-37b4bd7c9e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:04.284510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1158c-f9c6-4747-aa5e-37b4bd7c9e45 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:04.284645 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1158c-f9c6-4747-aa5e-37b4bd7c9e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:04.285000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1158c-f9c6-4747-aa5e-37b4bd7c9e45 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:04.286291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1158c-f9c6-4747-aa5e-37b4bd7c9e45 None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:04.287833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1158c-f9c6-4747-aa5e-37b4bd7c9e45 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:04.290007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 198/372] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:56:04 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:04.290878 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1158c-f9c6-4747-aa5e-37b4bd7c9e45 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:04.314913 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bba49-d771-4a02-a4ab-d8a2de6f3b45 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:04.330345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:04.355862 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71008d2c-3bf6-4c7b-a70b-03cf6fd4a992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:04.357065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71008d2c-3bf6-4c7b-a70b-03cf6fd4a992 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:04.357524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71008d2c-3bf6-4c7b-a70b-03cf6fd4a992 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:04.357996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71008d2c-3bf6-4c7b-a70b-03cf6fd4a992 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:04.359436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71008d2c-3bf6-4c7b-a70b-03cf6fd4a992 None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:04.361187 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71008d2c-3bf6-4c7b-a70b-03cf6fd4a992 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:04.363384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 175/373] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:56:04 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:04.364402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71008d2c-3bf6-4c7b-a70b-03cf6fd4a992 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:04.408190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1158c-f9c6-4747-aa5e-37b4bd7c9e45 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:04.449518 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c56d6-e93a-4cbd-9ff5-4e6e5751f966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:04.450484 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c56d6-e93a-4cbd-9ff5-4e6e5751f966 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:04.450844 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c56d6-e93a-4cbd-9ff5-4e6e5751f966 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:04.451193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c56d6-e93a-4cbd-9ff5-4e6e5751f966 None admin] PATH_INFO: `/v3/projects/a27dd5dc3697487bbb9b172396ff08c2` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:04.452544 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c56d6-e93a-4cbd-9ff5-4e6e5751f966 None admin] RBAC: Authorizing `identity:get_project(project_id=a27dd5dc3697487bbb9b172396ff08c2)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:04.454151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c56d6-e93a-4cbd-9ff5-4e6e5751f966 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:04.456836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 199/374] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:56:04 2022] GET /identity/v3/projects/a27dd5dc3697487bbb9b172396ff08c2 => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:04.457742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c56d6-e93a-4cbd-9ff5-4e6e5751f966 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:04.488649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71008d2c-3bf6-4c7b-a70b-03cf6fd4a992 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:04.496909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:04.527997 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e183f2e-8542-4426-b215-7412fbd719a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:04.529228 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e183f2e-8542-4426-b215-7412fbd719a1 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:04.529710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e183f2e-8542-4426-b215-7412fbd719a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:04.530181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e183f2e-8542-4426-b215-7412fbd719a1 None admin] PATH_INFO: `/v3/groups/138bdc814b3c4eb9885c7cb5da450386` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:04.531651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e183f2e-8542-4426-b215-7412fbd719a1 None admin] RBAC: Authorizing `identity:get_group(group_id=138bdc814b3c4eb9885c7cb5da450386)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:04.533559 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e183f2e-8542-4426-b215-7412fbd719a1 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:04.535456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 176/375] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jul 21 08:56:04 2022] GET /identity/v3/groups/138bdc814b3c4eb9885c7cb5da450386 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:04.537284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e183f2e-8542-4426-b215-7412fbd719a1 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:04.567476 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c56d6-e93a-4cbd-9ff5-4e6e5751f966 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:04.592748 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:04.605961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d66a5f-9b9f-4962-9ae3-021462317e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:04.606831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d66a5f-9b9f-4962-9ae3-021462317e27 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:04.607114 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d66a5f-9b9f-4962-9ae3-021462317e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:04.608569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d66a5f-9b9f-4962-9ae3-021462317e27 None admin] PATH_INFO: `/v3/groups/138bdc814b3c4eb9885c7cb5da450386` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:04.610534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d66a5f-9b9f-4962-9ae3-021462317e27 None admin] RBAC: Authorizing `identity:get_group(group_id=138bdc814b3c4eb9885c7cb5da450386)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:04.612041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d66a5f-9b9f-4962-9ae3-021462317e27 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:04.614348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 200/376] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jul 21 08:56:04 2022] GET /identity/v3/groups/138bdc814b3c4eb9885c7cb5da450386 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:04.614893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d66a5f-9b9f-4962-9ae3-021462317e27 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:04.659936 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e183f2e-8542-4426-b215-7412fbd719a1 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:04.698698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6168ce0-f2a1-43d6-9e0d-163e27b2c25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:04.700564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6168ce0-f2a1-43d6-9e0d-163e27b2c25e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:04.701060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6168ce0-f2a1-43d6-9e0d-163e27b2c25e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:04.701529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6168ce0-f2a1-43d6-9e0d-163e27b2c25e None admin] PATH_INFO: `/v3/groups/2cda83ee8d9e485c8e511e52510ff97f` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:04.710826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6168ce0-f2a1-43d6-9e0d-163e27b2c25e None admin] RBAC: Authorizing `identity:get_group(group_id=2cda83ee8d9e485c8e511e52510ff97f)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:04.712698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6168ce0-f2a1-43d6-9e0d-163e27b2c25e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:04.715800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 177/377] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jul 21 08:56:04 2022] GET /identity/v3/groups/2cda83ee8d9e485c8e511e52510ff97f => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:04.716840 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6168ce0-f2a1-43d6-9e0d-163e27b2c25e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:04.735346 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d66a5f-9b9f-4962-9ae3-021462317e27 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:04.737894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:04.775670 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cac6fc-6aed-47f6-bc2a-754f04b82353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:04.776330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cac6fc-6aed-47f6-bc2a-754f04b82353 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:04.776618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cac6fc-6aed-47f6-bc2a-754f04b82353 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:04.776898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cac6fc-6aed-47f6-bc2a-754f04b82353 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:04.778440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cac6fc-6aed-47f6-bc2a-754f04b82353 None admin] RBAC: Adding query filter params (group.id=138bdc814b3c4eb9885c7cb5da450386, role.id=3d7c80cf23864f58bf21f893ba7691d6, scope.project.id=551f963b02f4411395171476c9cc2d68) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:04.778753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cac6fc-6aed-47f6-bc2a-754f04b82353 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:04.780605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cac6fc-6aed-47f6-bc2a-754f04b82353 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:04.786770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 201/378] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jul 21 08:56:04 2022] GET /identity/v3/role_assignments?group.id=138bdc814b3c4eb9885c7cb5da450386&scope.project.id=551f963b02f4411395171476c9cc2d68&role.id=3d7c80cf23864f58bf21f893ba7691d6 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:04.788659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cac6fc-6aed-47f6-bc2a-754f04b82353 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:04.848467 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6168ce0-f2a1-43d6-9e0d-163e27b2c25e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:04.876404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:04.891612 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b1c99-5204-49c2-a871-b8bb0a10e195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:04.892869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b1c99-5204-49c2-a871-b8bb0a10e195 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:04.893376 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b1c99-5204-49c2-a871-b8bb0a10e195 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:04.893919 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b1c99-5204-49c2-a871-b8bb0a10e195 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:04.895041 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b1c99-5204-49c2-a871-b8bb0a10e195 None admin] RBAC: Adding query filter params (group.id=138bdc814b3c4eb9885c7cb5da450386, role.id=5a989bb8c8a1465ca76622ffb504dee5, scope.project.id=551f963b02f4411395171476c9cc2d68) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:04.895660 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b1c99-5204-49c2-a871-b8bb0a10e195 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:04.897526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b1c99-5204-49c2-a871-b8bb0a10e195 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:04.906519 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 178/379] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jul 21 08:56:04 2022] GET /identity/v3/role_assignments?group.id=138bdc814b3c4eb9885c7cb5da450386&scope.project.id=551f963b02f4411395171476c9cc2d68&role.id=5a989bb8c8a1465ca76622ffb504dee5 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:04.909773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cac6fc-6aed-47f6-bc2a-754f04b82353 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:04.952087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2fe91-4e94-4c2c-ab83-5d9838514d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:04.952772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2fe91-4e94-4c2c-ab83-5d9838514d7b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:04.953057 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2fe91-4e94-4c2c-ab83-5d9838514d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:04.953335 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2fe91-4e94-4c2c-ab83-5d9838514d7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:04.954256 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2fe91-4e94-4c2c-ab83-5d9838514d7b None admin] RBAC: Adding query filter params (group.id=2cda83ee8d9e485c8e511e52510ff97f, role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=a27dd5dc3697487bbb9b172396ff08c2) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:04.954577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2fe91-4e94-4c2c-ab83-5d9838514d7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:04.956074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d2fe91-4e94-4c2c-ab83-5d9838514d7b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:04.963435 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 202/380] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jul 21 08:56:04 2022] GET /identity/v3/role_assignments?group.id=2cda83ee8d9e485c8e511e52510ff97f&scope.project.id=a27dd5dc3697487bbb9b172396ff08c2&role.id=000e942517f2469ba36d2e65978bc314 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:05.049052 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa02686-ebe5-4f39-84d7-4e27552e4ab7 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:05.049430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa02686-ebe5-4f39-84d7-4e27552e4ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:05.049638 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa02686-ebe5-4f39-84d7-4e27552e4ab7 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:05.051496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 179/381] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:05.066226 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af9b46-1963-425a-b3d8-4f9f8daba935 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:05.066555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af9b46-1963-425a-b3d8-4f9f8daba935 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:05.066802 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af9b46-1963-425a-b3d8-4f9f8daba935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:05.093306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a6af9b46-1963-425a-b3d8-4f9f8daba935 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:05.094568 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a6af9b46-1963-425a-b3d8-4f9f8daba935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:05.127372 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:05.157387 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb156554-5150-4fa5-b8bf-d5d6c07a16d6 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:05.158186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb156554-5150-4fa5-b8bf-d5d6c07a16d6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:05.158720 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb156554-5150-4fa5-b8bf-d5d6c07a16d6 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:05.160775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 180/382] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:05.167233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05060292-a041-44b4-9215-c6da7f7f9ee4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:05.167849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05060292-a041-44b4-9215-c6da7f7f9ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:05.168516 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05060292-a041-44b4-9215-c6da7f7f9ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:05.187826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 203/383] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:05.193967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-05060292-a041-44b4-9215-c6da7f7f9ee4 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:05.195208 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-05060292-a041-44b4-9215-c6da7f7f9ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:05.218981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af9b46-1963-425a-b3d8-4f9f8daba935 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:05.291086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 181/384] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:05.322687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05060292-a041-44b4-9215-c6da7f7f9ee4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:05.334340 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af9b46-1963-425a-b3d8-4f9f8daba935 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:05.374483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc715cde-b662-41c3-911a-ddf5dcec3841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:05.375309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc715cde-b662-41c3-911a-ddf5dcec3841 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:05.375593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc715cde-b662-41c3-911a-ddf5dcec3841 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:05.375867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc715cde-b662-41c3-911a-ddf5dcec3841 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:05.387953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc715cde-b662-41c3-911a-ddf5dcec3841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-cc715cde-b662-41c3-911a-ddf5dcec3841 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:05.396388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:05.397331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:05.398101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:05.398101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:05.398101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:05.398101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:05.398101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:05.398101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:05.398101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:05.398101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:05.398101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:05.398101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:05.398101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:56:05.398101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:05.402981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 204/385] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:56:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:05.404562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc715cde-b662-41c3-911a-ddf5dcec3841 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:05.432094 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:05.483982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05060292-a041-44b4-9215-c6da7f7f9ee4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:05.500025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:05.520648 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc715cde-b662-41c3-911a-ddf5dcec3841 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:05.559781 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f191314-b914-44e5-bd1d-cae25fb03489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:05.560745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f191314-b914-44e5-bd1d-cae25fb03489 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:05.561032 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f191314-b914-44e5-bd1d-cae25fb03489 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:05.561363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f191314-b914-44e5-bd1d-cae25fb03489 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:05.562137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f191314-b914-44e5-bd1d-cae25fb03489 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:05.562449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f191314-b914-44e5-bd1d-cae25fb03489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:05.563859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f191314-b914-44e5-bd1d-cae25fb03489 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:05.568677 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66127c-2aa9-4be5-9e26-20249746f0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:05.569507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66127c-2aa9-4be5-9e26-20249746f0fd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:05.569796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66127c-2aa9-4be5-9e26-20249746f0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:05.570079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66127c-2aa9-4be5-9e26-20249746f0fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:05.580125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 205/386] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:05.581560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f191314-b914-44e5-bd1d-cae25fb03489 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:05.604594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66127c-2aa9-4be5-9e26-20249746f0fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-ed66127c-2aa9-4be5-9e26-20249746f0fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:05.613038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:05.613971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:05.614747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:05.614747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:05.614747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:05.614747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:05.614747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:05.614747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:05.614747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:05.614747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:05.614747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:05.614747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:05.614747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:56:05.614747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:05.614747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 182/387] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:05.620213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66127c-2aa9-4be5-9e26-20249746f0fd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:05.725784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:05.769132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f191314-b914-44e5-bd1d-cae25fb03489 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:05.792888 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66127c-2aa9-4be5-9e26-20249746f0fd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:05.845928 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b714d67-c250-4db9-b22e-a785a63fe89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:05.846683 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b714d67-c250-4db9-b22e-a785a63fe89e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:05.846978 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b714d67-c250-4db9-b22e-a785a63fe89e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:05.847260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b714d67-c250-4db9-b22e-a785a63fe89e None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:05.848069 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b714d67-c250-4db9-b22e-a785a63fe89e None admin] RBAC: Adding query filter params (name=member) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:05.848383 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b714d67-c250-4db9-b22e-a785a63fe89e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:05.850597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b714d67-c250-4db9-b22e-a785a63fe89e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:05.860686 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 183/388] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:56:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:05.869143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf060a-b57e-44fd-a65b-b27a2131fee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:05.869480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b714d67-c250-4db9-b22e-a785a63fe89e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:05.870037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf060a-b57e-44fd-a65b-b27a2131fee2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:05.870320 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf060a-b57e-44fd-a65b-b27a2131fee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:05.870593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf060a-b57e-44fd-a65b-b27a2131fee2 None admin] PATH_INFO: `/v3/groups/138bdc814b3c4eb9885c7cb5da450386` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:05.876764 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf060a-b57e-44fd-a65b-b27a2131fee2 None admin] RBAC: Authorizing `identity:get_group(group_id=138bdc814b3c4eb9885c7cb5da450386)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:05.878250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf060a-b57e-44fd-a65b-b27a2131fee2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:05.880176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 206/389] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jul 21 08:56:05 2022] GET /identity/v3/groups/138bdc814b3c4eb9885c7cb5da450386 => generated 226 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:05.886008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf060a-b57e-44fd-a65b-b27a2131fee2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:05.914902 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:06.025532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b714d67-c250-4db9-b22e-a785a63fe89e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:06.081386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d121f-d651-464d-a2a3-6ff02f733cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:06.082261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d121f-d651-464d-a2a3-6ff02f733cdb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:06.082547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d121f-d651-464d-a2a3-6ff02f733cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:06.082822 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d121f-d651-464d-a2a3-6ff02f733cdb None admin] PATH_INFO: `/v3/groups/138bdc814b3c4eb9885c7cb5da450386` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:06.084127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d121f-d651-464d-a2a3-6ff02f733cdb None admin] RBAC: Authorizing `identity:get_group(group_id=138bdc814b3c4eb9885c7cb5da450386)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:06.086061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d121f-d651-464d-a2a3-6ff02f733cdb None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:06.088142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 184/390] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jul 21 08:56:05 2022] GET /identity/v3/groups/138bdc814b3c4eb9885c7cb5da450386 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:06.099171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d121f-d651-464d-a2a3-6ff02f733cdb None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:06.114183 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf060a-b57e-44fd-a65b-b27a2131fee2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:06.160353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd1ab0-e29e-4f33-a589-05bc55d95931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:06.161210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd1ab0-e29e-4f33-a589-05bc55d95931 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:06.161494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd1ab0-e29e-4f33-a589-05bc55d95931 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:06.161763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd1ab0-e29e-4f33-a589-05bc55d95931 None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:06.163112 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd1ab0-e29e-4f33-a589-05bc55d95931 None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:06.164679 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd1ab0-e29e-4f33-a589-05bc55d95931 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:06.166682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 207/391] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:56:05 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:06.173755 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd1ab0-e29e-4f33-a589-05bc55d95931 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:06.221018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:06.305660 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d121f-d651-464d-a2a3-6ff02f733cdb None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:06.316168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:06.362767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7f41f-f309-4ca2-83bf-d7a1bb955738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:06.363885 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7f41f-f309-4ca2-83bf-d7a1bb955738 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:06.364261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7f41f-f309-4ca2-83bf-d7a1bb955738 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:06.364555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7f41f-f309-4ca2-83bf-d7a1bb955738 None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:06.366481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7f41f-f309-4ca2-83bf-d7a1bb955738 None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:06.368513 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7f41f-f309-4ca2-83bf-d7a1bb955738 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:06.371829 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 185/392] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:56:06 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:06.379584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7f41f-f309-4ca2-83bf-d7a1bb955738 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:06.380527 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd1ab0-e29e-4f33-a589-05bc55d95931 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:06.453077 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96800d9f-d713-451f-8082-42cd17bb2a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:06.454228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96800d9f-d713-451f-8082-42cd17bb2a29 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:06.454518 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96800d9f-d713-451f-8082-42cd17bb2a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:06.455689 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96800d9f-d713-451f-8082-42cd17bb2a29 None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431/groups/138bdc814b3c4eb9885c7cb5da450386/roles/5a989bb8c8a1465ca76622ffb504dee5` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:06.462883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96800d9f-d713-451f-8082-42cd17bb2a29 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1099d961ac5441c973dfbd04812f431, group_id=138bdc814b3c4eb9885c7cb5da450386, role_id=5a989bb8c8a1465ca76622ffb504dee5)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:06.464405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96800d9f-d713-451f-8082-42cd17bb2a29 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:06.489829 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 208/393] 10.222.0.13 () {58 vars in 1442 bytes} [Thu Jul 21 08:56:06 2022] PUT /identity/v3/projects/e1099d961ac5441c973dfbd04812f431/groups/138bdc814b3c4eb9885c7cb5da450386/roles/5a989bb8c8a1465ca76622ffb504dee5 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:56:06.566248 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7f41f-f309-4ca2-83bf-d7a1bb955738 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:06.620241 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69f0e8-b300-4f88-8d6e-02070eaf6059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:06.621206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69f0e8-b300-4f88-8d6e-02070eaf6059 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:06.622004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69f0e8-b300-4f88-8d6e-02070eaf6059 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:06.622352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69f0e8-b300-4f88-8d6e-02070eaf6059 None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431/groups/138bdc814b3c4eb9885c7cb5da450386/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:06.629701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69f0e8-b300-4f88-8d6e-02070eaf6059 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1099d961ac5441c973dfbd04812f431, group_id=138bdc814b3c4eb9885c7cb5da450386, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:06.631210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69f0e8-b300-4f88-8d6e-02070eaf6059 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:06.661120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 186/394] 10.222.0.13 () {58 vars in 1442 bytes} [Thu Jul 21 08:56:06 2022] PUT /identity/v3/projects/e1099d961ac5441c973dfbd04812f431/groups/138bdc814b3c4eb9885c7cb5da450386/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:56:07.358220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52814722-07bd-4201-968d-08a43effb36b None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:07.358731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52814722-07bd-4201-968d-08a43effb36b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:07.358831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52814722-07bd-4201-968d-08a43effb36b None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:07.360499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 209/395] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:07.365432 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d377cc-b7f6-43d2-88f3-850bf022613a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:07.365862 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d377cc-b7f6-43d2-88f3-850bf022613a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:07.366036 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d377cc-b7f6-43d2-88f3-850bf022613a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:07.393015 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b1d377cc-b7f6-43d2-88f3-850bf022613a None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:07.394252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b1d377cc-b7f6-43d2-88f3-850bf022613a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:07.416626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:07.471044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 187/396] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:07.492939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52814722-07bd-4201-968d-08a43effb36b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:07.609095 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:07.642741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52814722-07bd-4201-968d-08a43effb36b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:07.688784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc42b6-ab50-4b90-906e-4bcc6f9062d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:07.690041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc42b6-ab50-4b90-906e-4bcc6f9062d3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:07.690528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc42b6-ab50-4b90-906e-4bcc6f9062d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:07.691023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc42b6-ab50-4b90-906e-4bcc6f9062d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:07.712176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc42b6-ab50-4b90-906e-4bcc6f9062d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-11bc42b6-ab50-4b90-906e-4bcc6f9062d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:07.721709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:07.722582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:07.723350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:07.723350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:07.723350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:07.723350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:07.723350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:07.723350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:07.723350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:07.723350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:07.723350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:07.723350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:07.723350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:56:07.723350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:07.723350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 210/397] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:07.728061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3795143-826d-41ca-8fc5-0c00bdd95a47 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:07.728563 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc42b6-ab50-4b90-906e-4bcc6f9062d3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:07.728852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3795143-826d-41ca-8fc5-0c00bdd95a47 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:07.729426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3795143-826d-41ca-8fc5-0c00bdd95a47 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:07.731652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 188/398] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:07.737537 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e77580-79f1-4c55-b261-3c3f6f7cb1f6 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:07.738683 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e77580-79f1-4c55-b261-3c3f6f7cb1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:07.738683 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e77580-79f1-4c55-b261-3c3f6f7cb1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:07.781837 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d5e77580-79f1-4c55-b261-3c3f6f7cb1f6 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:07.782494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d5e77580-79f1-4c55-b261-3c3f6f7cb1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:07.864314 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc42b6-ab50-4b90-906e-4bcc6f9062d3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:07.886598 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 189/399] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:56:07.910223 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e77580-79f1-4c55-b261-3c3f6f7cb1f6 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:07.911436 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2329a-3217-4ebd-ba17-adef3b3c2382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:07.912352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2329a-3217-4ebd-ba17-adef3b3c2382 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:07.912832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2329a-3217-4ebd-ba17-adef3b3c2382 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:07.913272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2329a-3217-4ebd-ba17-adef3b3c2382 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:07.914273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2329a-3217-4ebd-ba17-adef3b3c2382 None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:07.914709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2329a-3217-4ebd-ba17-adef3b3c2382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:07.916266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2329a-3217-4ebd-ba17-adef3b3c2382 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:07.927400 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 211/400] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:56:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:07.932666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2329a-3217-4ebd-ba17-adef3b3c2382 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:07.937989 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:08.051963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:08.065062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e77580-79f1-4c55-b261-3c3f6f7cb1f6 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:08.102437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de912d-e1bc-4065-83ba-071260b3c2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:08.103255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de912d-e1bc-4065-83ba-071260b3c2ac None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:08.103541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de912d-e1bc-4065-83ba-071260b3c2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:08.103818 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de912d-e1bc-4065-83ba-071260b3c2ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:08.125345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de912d-e1bc-4065-83ba-071260b3c2ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-79de912d-e1bc-4065-83ba-071260b3c2ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:08.133657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:08.134561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:08.135450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:08.135450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:08.135450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:08.135450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:08.135450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:08.135450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:08.135450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:08.135450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:08.135450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:08.135450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:08.135450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:56:08.135450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:08.135450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 190/401] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:56:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:08.136588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867ef07-c361-42a0-94ad-93073d6d6b73 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:08.136957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867ef07-c361-42a0-94ad-93073d6d6b73 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:08.137758 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867ef07-c361-42a0-94ad-93073d6d6b73 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:08.143208 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 191/402] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:08.143208 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867ef07-c361-42a0-94ad-93073d6d6b73 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:08.158821 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2329a-3217-4ebd-ba17-adef3b3c2382 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:08.207964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518a6a6-93c0-4e9e-990f-e248517de76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:08.207964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518a6a6-93c0-4e9e-990f-e248517de76b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:08.207964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518a6a6-93c0-4e9e-990f-e248517de76b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:08.207964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518a6a6-93c0-4e9e-990f-e248517de76b None admin] PATH_INFO: `/v3/groups/138bdc814b3c4eb9885c7cb5da450386` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:08.208385 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518a6a6-93c0-4e9e-990f-e248517de76b None admin] RBAC: Authorizing `identity:get_group(group_id=138bdc814b3c4eb9885c7cb5da450386)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:08.209780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518a6a6-93c0-4e9e-990f-e248517de76b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:08.211740 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 212/403] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jul 21 08:56:07 2022] GET /identity/v3/groups/138bdc814b3c4eb9885c7cb5da450386 => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:08.213166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d389dc-0bdc-4977-839a-62ab7ece7787 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:08.213717 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d389dc-0bdc-4977-839a-62ab7ece7787 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:08.213833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d389dc-0bdc-4977-839a-62ab7ece7787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:08.230889 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d1d389dc-0bdc-4977-839a-62ab7ece7787 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:08.232505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d1d389dc-0bdc-4977-839a-62ab7ece7787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:08.261161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867ef07-c361-42a0-94ad-93073d6d6b73 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:08.303030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa72867-b043-43a1-a718-e0e8eea664f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:08.303844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa72867-b043-43a1-a718-e0e8eea664f2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:08.304154 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa72867-b043-43a1-a718-e0e8eea664f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:08.304458 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa72867-b043-43a1-a718-e0e8eea664f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:08.305324 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa72867-b043-43a1-a718-e0e8eea664f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:08.305637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa72867-b043-43a1-a718-e0e8eea664f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:08.307103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa72867-b043-43a1-a718-e0e8eea664f2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:08.314216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:08.320730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 192/405] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:08.321795 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa72867-b043-43a1-a718-e0e8eea664f2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:08.324192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 213/405] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:08.327087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d389dc-0bdc-4977-839a-62ab7ece7787 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:08.336435 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:08.443775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa72867-b043-43a1-a718-e0e8eea664f2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:08.453885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d389dc-0bdc-4977-839a-62ab7ece7787 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:08.481976 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7351c-b119-4a5c-ab4b-42e6c927a568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:08.482837 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7351c-b119-4a5c-ab4b-42e6c927a568 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:08.483224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7351c-b119-4a5c-ab4b-42e6c927a568 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:08.483563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7351c-b119-4a5c-ab4b-42e6c927a568 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:08.484969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7351c-b119-4a5c-ab4b-42e6c927a568 None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:08.486559 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7351c-b119-4a5c-ab4b-42e6c927a568 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:08.488971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 193/406] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:56:08 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:08.490187 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7351c-b119-4a5c-ab4b-42e6c927a568 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:08.491637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cf632-542e-42ff-b330-b0e7578aaa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:08.492548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cf632-542e-42ff-b330-b0e7578aaa38 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:08.492948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cf632-542e-42ff-b330-b0e7578aaa38 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:08.493335 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cf632-542e-42ff-b330-b0e7578aaa38 None admin] PATH_INFO: `/v3/groups/138bdc814b3c4eb9885c7cb5da450386` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:08.494517 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394cf632-542e-42ff-b330-b0e7578aaa38 None admin] RBAC: Authorizing `identity:get_group(group_id=138bdc814b3c4eb9885c7cb5da450386)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:08.496176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394cf632-542e-42ff-b330-b0e7578aaa38 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:08.498922 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 214/407] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jul 21 08:56:08 2022] GET /identity/v3/groups/138bdc814b3c4eb9885c7cb5da450386 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:08.499690 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cf632-542e-42ff-b330-b0e7578aaa38 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:08.573544 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:08.577936 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:08.604576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7351c-b119-4a5c-ab4b-42e6c927a568 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:08.615708 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cf632-542e-42ff-b330-b0e7578aaa38 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:08.643293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95808b1a-1827-419c-855b-c972d1dd47bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:08.644315 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95808b1a-1827-419c-855b-c972d1dd47bf None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:08.644669 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95808b1a-1827-419c-855b-c972d1dd47bf None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:08.645014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95808b1a-1827-419c-855b-c972d1dd47bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:08.656381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899db5cc-3caf-4d50-96a6-4d0e65447b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:08.657275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899db5cc-3caf-4d50-96a6-4d0e65447b45 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:08.657562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899db5cc-3caf-4d50-96a6-4d0e65447b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:08.657831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899db5cc-3caf-4d50-96a6-4d0e65447b45 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68/groups/138bdc814b3c4eb9885c7cb5da450386/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:08.658212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95808b1a-1827-419c-855b-c972d1dd47bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:08.659852 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899db5cc-3caf-4d50-96a6-4d0e65447b45 None admin] RBAC: Authorizing `identity:create_grant(project_id=551f963b02f4411395171476c9cc2d68, group_id=138bdc814b3c4eb9885c7cb5da450386, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:08.661335 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899db5cc-3caf-4d50-96a6-4d0e65447b45 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-95808b1a-1827-419c-855b-c972d1dd47bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:08.666817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:08.667806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:08.669233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:08.669233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:08.669233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:08.669233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:08.669233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:08.669233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:08.669233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:08.669233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:08.669233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:08.669233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:08.669233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:08.669233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:08.669233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 194/408] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:56:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:08.675504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95808b1a-1827-419c-855b-c972d1dd47bf None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:08.682541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 215/409] 10.222.0.13 () {58 vars in 1442 bytes} [Thu Jul 21 08:56:08 2022] PUT /identity/v3/projects/551f963b02f4411395171476c9cc2d68/groups/138bdc814b3c4eb9885c7cb5da450386/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:56:08.683581 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899db5cc-3caf-4d50-96a6-4d0e65447b45 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:08.830112 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899db5cc-3caf-4d50-96a6-4d0e65447b45 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:08.868457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95808b1a-1827-419c-855b-c972d1dd47bf None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:08.909992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:08.921462 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e3705-170e-404c-94ea-98248aee8e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:08.922148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e3705-170e-404c-94ea-98248aee8e0b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:08.922426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e3705-170e-404c-94ea-98248aee8e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:08.922695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e3705-170e-404c-94ea-98248aee8e0b None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:08.923454 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e3705-170e-404c-94ea-98248aee8e0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:08.923755 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e3705-170e-404c-94ea-98248aee8e0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:08.925261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e3705-170e-404c-94ea-98248aee8e0b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:08.939044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 216/410] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:08.944729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2987383-b1e5-4859-a009-28045a520ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:08.945492 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e3705-170e-404c-94ea-98248aee8e0b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:08.945608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2987383-b1e5-4859-a009-28045a520ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:08.945889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2987383-b1e5-4859-a009-28045a520ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:08.946272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2987383-b1e5-4859-a009-28045a520ff9 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:08.947541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2987383-b1e5-4859-a009-28045a520ff9 None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:08.949261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2987383-b1e5-4859-a009-28045a520ff9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:08.951667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 195/411] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:56:08 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:08.958238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2987383-b1e5-4859-a009-28045a520ff9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:08.964275 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:09.083488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2987383-b1e5-4859-a009-28045a520ff9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:09.148309 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4a556-72e8-4eab-be48-4cb66b4b4829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:09.149590 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4a556-72e8-4eab-be48-4cb66b4b4829 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:09.150114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4a556-72e8-4eab-be48-4cb66b4b4829 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:09.150591 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4a556-72e8-4eab-be48-4cb66b4b4829 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68/groups/138bdc814b3c4eb9885c7cb5da450386/roles/5a989bb8c8a1465ca76622ffb504dee5` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:09.154935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e3705-170e-404c-94ea-98248aee8e0b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:09.158203 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4a556-72e8-4eab-be48-4cb66b4b4829 None admin] RBAC: Authorizing `identity:create_grant(project_id=551f963b02f4411395171476c9cc2d68, group_id=138bdc814b3c4eb9885c7cb5da450386, role_id=5a989bb8c8a1465ca76622ffb504dee5)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:09.160044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4a556-72e8-4eab-be48-4cb66b4b4829 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:09.190601 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 196/412] 10.222.0.13 () {58 vars in 1442 bytes} [Thu Jul 21 08:56:08 2022] PUT /identity/v3/projects/551f963b02f4411395171476c9cc2d68/groups/138bdc814b3c4eb9885c7cb5da450386/roles/5a989bb8c8a1465ca76622ffb504dee5 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:56:09.205103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f6f05-51ef-4bae-805d-07455e3e4aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:09.206006 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f6f05-51ef-4bae-805d-07455e3e4aca None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:09.206285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f6f05-51ef-4bae-805d-07455e3e4aca None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:09.206556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f6f05-51ef-4bae-805d-07455e3e4aca None admin] PATH_INFO: `/v3/groups/2cda83ee8d9e485c8e511e52510ff97f` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:09.208887 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f6f05-51ef-4bae-805d-07455e3e4aca None admin] RBAC: Authorizing `identity:get_group(group_id=2cda83ee8d9e485c8e511e52510ff97f)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:09.210465 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f6f05-51ef-4bae-805d-07455e3e4aca None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:09.217547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 217/413] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jul 21 08:56:08 2022] GET /identity/v3/groups/2cda83ee8d9e485c8e511e52510ff97f => generated 229 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:09.219112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4a556-72e8-4eab-be48-4cb66b4b4829 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:09.276541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a085ea0-faf4-45f0-9a58-6f521ec510d4 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:09.276892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a085ea0-faf4-45f0-9a58-6f521ec510d4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:09.277101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a085ea0-faf4-45f0-9a58-6f521ec510d4 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:09.279472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 218/414] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:56:09.285315 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c657b-56ea-4f38-a4b7-8ad7a779196c None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:09.285615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c657b-56ea-4f38-a4b7-8ad7a779196c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:09.285873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c657b-56ea-4f38-a4b7-8ad7a779196c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:09.317296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7f4c657b-56ea-4f38-a4b7-8ad7a779196c None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:09.318557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7f4c657b-56ea-4f38-a4b7-8ad7a779196c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:09.366845 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:09.430746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4a556-72e8-4eab-be48-4cb66b4b4829 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:09.432107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:09.450358 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 219/415] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:56:09.451752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd234f9-c881-46a5-a4ac-963f2c7810b5 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:09.459755 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd234f9-c881-46a5-a4ac-963f2c7810b5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:09.460407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd234f9-c881-46a5-a4ac-963f2c7810b5 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:09.462011 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 220/416] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:09 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:09.469472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96677ad1-ab63-4bed-9c27-f4fee163e09b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:09.469787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96677ad1-ab63-4bed-9c27-f4fee163e09b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:09.470027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96677ad1-ab63-4bed-9c27-f4fee163e09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:09.486128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12089778-a812-4001-b19c-da8b5d26815f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:09.487745 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12089778-a812-4001-b19c-da8b5d26815f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:09.488292 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12089778-a812-4001-b19c-da8b5d26815f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:09.488763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12089778-a812-4001-b19c-da8b5d26815f None admin] PATH_INFO: `/v3/projects/a27dd5dc3697487bbb9b172396ff08c2` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:09.490322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12089778-a812-4001-b19c-da8b5d26815f None admin] RBAC: Authorizing `identity:get_project(project_id=a27dd5dc3697487bbb9b172396ff08c2)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:09.491899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12089778-a812-4001-b19c-da8b5d26815f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:09.494074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 197/417] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:56:09 2022] GET /identity/v3/projects/a27dd5dc3697487bbb9b172396ff08c2 => generated 344 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:09.494957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12089778-a812-4001-b19c-da8b5d26815f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:09.498419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-96677ad1-ab63-4bed-9c27-f4fee163e09b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:09.500122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-96677ad1-ab63-4bed-9c27-f4fee163e09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:09.598253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 221/418] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:09.599389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96677ad1-ab63-4bed-9c27-f4fee163e09b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:09.648521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12089778-a812-4001-b19c-da8b5d26815f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:09.686374 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:09.693697 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f64f0-378a-45bb-bb0f-9137bbc032bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:09.694682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f64f0-378a-45bb-bb0f-9137bbc032bd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:09.695043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f64f0-378a-45bb-bb0f-9137bbc032bd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:09.695500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f64f0-378a-45bb-bb0f-9137bbc032bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:09.707856 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f64f0-378a-45bb-bb0f-9137bbc032bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-5a4f64f0-378a-45bb-bb0f-9137bbc032bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:09.716489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:09.717668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:09.718585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:09.718585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:09.718585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:09.718585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:09.718585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:09.718585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:09.718585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:09.718585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:09.718585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:09.718585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:09.718585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:56:09.718585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:09.722737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 198/419] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:09.723071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96677ad1-ab63-4bed-9c27-f4fee163e09b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:09.723657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f64f0-378a-45bb-bb0f-9137bbc032bd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:09.756377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:09.764795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb128a1-794c-43d9-ab3c-b1196c955a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:09.765932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb128a1-794c-43d9-ab3c-b1196c955a34 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:09.766418 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb128a1-794c-43d9-ab3c-b1196c955a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:09.766837 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb128a1-794c-43d9-ab3c-b1196c955a34 None admin] PATH_INFO: `/v3/projects/a27dd5dc3697487bbb9b172396ff08c2/groups/2cda83ee8d9e485c8e511e52510ff97f/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:09.769674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb128a1-794c-43d9-ab3c-b1196c955a34 None admin] RBAC: Authorizing `identity:create_grant(project_id=a27dd5dc3697487bbb9b172396ff08c2, group_id=2cda83ee8d9e485c8e511e52510ff97f, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:09.771908 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb128a1-794c-43d9-ab3c-b1196c955a34 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:09.790281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 222/420] 10.222.0.13 () {58 vars in 1442 bytes} [Thu Jul 21 08:56:09 2022] PUT /identity/v3/projects/a27dd5dc3697487bbb9b172396ff08c2/groups/2cda83ee8d9e485c8e511e52510ff97f/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:56:09.791583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb128a1-794c-43d9-ab3c-b1196c955a34 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:09.847755 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f64f0-378a-45bb-bb0f-9137bbc032bd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:09.896272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee19aad-abdf-45f0-9621-49595d3be051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:09.897230 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee19aad-abdf-45f0-9621-49595d3be051 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:09.897520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee19aad-abdf-45f0-9621-49595d3be051 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:09.897796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee19aad-abdf-45f0-9621-49595d3be051 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:09.921797 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee19aad-abdf-45f0-9621-49595d3be051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:09.936804 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb128a1-794c-43d9-ab3c-b1196c955a34 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-5ee19aad-abdf-45f0-9621-49595d3be051 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:09.940314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:09.941225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:09.941994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:09.941994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:09.941994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:09.941994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:09.941994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:09.941994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:09.941994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:09.941994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:09.941994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:09.941994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:09.941994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:56:09.941994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:09.944957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 199/421] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:56:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:09.951652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee19aad-abdf-45f0-9621-49595d3be051 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:10.008284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164399f-621c-48cb-ab6e-4368d3de0de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:10.008960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164399f-621c-48cb-ab6e-4368d3de0de4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:10.009291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164399f-621c-48cb-ab6e-4368d3de0de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:10.009818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164399f-621c-48cb-ab6e-4368d3de0de4 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:10.011059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164399f-621c-48cb-ab6e-4368d3de0de4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:10.011377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164399f-621c-48cb-ab6e-4368d3de0de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:10.013163 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164399f-621c-48cb-ab6e-4368d3de0de4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:10.020112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:10.027104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 223/422] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:56:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:10.032890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164399f-621c-48cb-ab6e-4368d3de0de4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:10.079252 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:10.089213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee19aad-abdf-45f0-9621-49595d3be051 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:10.135770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea028730-254a-495c-8e66-aa769f76eacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:10.136753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea028730-254a-495c-8e66-aa769f76eacd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:10.137745 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea028730-254a-495c-8e66-aa769f76eacd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:10.138136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea028730-254a-495c-8e66-aa769f76eacd None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:10.139002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea028730-254a-495c-8e66-aa769f76eacd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:10.139415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea028730-254a-495c-8e66-aa769f76eacd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:10.141138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea028730-254a-495c-8e66-aa769f76eacd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:10.151853 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 200/423] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:10.158680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea028730-254a-495c-8e66-aa769f76eacd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:10.196648 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164399f-621c-48cb-ab6e-4368d3de0de4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:10.282653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf44d63-945a-4cdd-a6f4-bc21aba15f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:10.283737 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf44d63-945a-4cdd-a6f4-bc21aba15f3a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:10.284283 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf44d63-945a-4cdd-a6f4-bc21aba15f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:10.284779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf44d63-945a-4cdd-a6f4-bc21aba15f3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:10.390085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea028730-254a-495c-8e66-aa769f76eacd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:10.406557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf44d63-945a-4cdd-a6f4-bc21aba15f3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:10.409398 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf44d63-945a-4cdd-a6f4-bc21aba15f3a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:10.422626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:10.480880 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634fe1b-72fa-49c6-bb24-002e7b7d955b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:10.481584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634fe1b-72fa-49c6-bb24-002e7b7d955b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:10.481871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634fe1b-72fa-49c6-bb24-002e7b7d955b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:10.482150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634fe1b-72fa-49c6-bb24-002e7b7d955b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:10.484874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:10.619483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 224/424] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 08:56:10.625471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634fe1b-72fa-49c6-bb24-002e7b7d955b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:10.628235 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634fe1b-72fa-49c6-bb24-002e7b7d955b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:10.637918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf44d63-945a-4cdd-a6f4-bc21aba15f3a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:10.753291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf44d63-945a-4cdd-a6f4-bc21aba15f3a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:10.790032 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a38fd-eb0c-44cd-8274-bceaab3c2bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:10.791273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a38fd-eb0c-44cd-8274-bceaab3c2bba None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:10.792048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a38fd-eb0c-44cd-8274-bceaab3c2bba None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:10.792545 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a38fd-eb0c-44cd-8274-bceaab3c2bba None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:10.794033 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a38fd-eb0c-44cd-8274-bceaab3c2bba None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:10.795786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a38fd-eb0c-44cd-8274-bceaab3c2bba None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:10.798011 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 225/425] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:56:10 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:10.803477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a38fd-eb0c-44cd-8274-bceaab3c2bba None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:10.814543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:10.839121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 201/426] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 08:56:10.847075 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634fe1b-72fa-49c6-bb24-002e7b7d955b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:10.876449 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:10.915163 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a38fd-eb0c-44cd-8274-bceaab3c2bba None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:10.950478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da9bef-530f-45b0-843a-86f02472c544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:10.951707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da9bef-530f-45b0-843a-86f02472c544 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:10.952219 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da9bef-530f-45b0-843a-86f02472c544 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:10.952703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da9bef-530f-45b0-843a-86f02472c544 None admin] PATH_INFO: `/v3/groups/138bdc814b3c4eb9885c7cb5da450386` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:10.954104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da9bef-530f-45b0-843a-86f02472c544 None admin] RBAC: Authorizing `identity:get_group(group_id=138bdc814b3c4eb9885c7cb5da450386)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:10.955772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da9bef-530f-45b0-843a-86f02472c544 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:10.957997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 226/427] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jul 21 08:56:10 2022] GET /identity/v3/groups/138bdc814b3c4eb9885c7cb5da450386 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:10.963794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da9bef-530f-45b0-843a-86f02472c544 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:11.060442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:11.072041 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634fe1b-72fa-49c6-bb24-002e7b7d955b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:11.102916 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da9bef-530f-45b0-843a-86f02472c544 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:11.151322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09f4fc-017a-440c-80e8-f634e74a9763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:11.152705 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09f4fc-017a-440c-80e8-f634e74a9763 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:11.153285 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09f4fc-017a-440c-80e8-f634e74a9763 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:11.153769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09f4fc-017a-440c-80e8-f634e74a9763 None admin] PATH_INFO: `/v3/projects/e1099d961ac5441c973dfbd04812f431` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:11.155434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09f4fc-017a-440c-80e8-f634e74a9763 None admin] RBAC: Authorizing `identity:get_project(project_id=e1099d961ac5441c973dfbd04812f431)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:11.158543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09f4fc-017a-440c-80e8-f634e74a9763 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:11.168697 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 202/428] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:56:10 2022] GET /identity/v3/projects/e1099d961ac5441c973dfbd04812f431 => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:11.169994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09f4fc-017a-440c-80e8-f634e74a9763 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:11.187635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08971cdf-4e3d-4518-bce5-9cadeceb06f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:11.188687 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08971cdf-4e3d-4518-bce5-9cadeceb06f4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:11.189187 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08971cdf-4e3d-4518-bce5-9cadeceb06f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:11.189668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08971cdf-4e3d-4518-bce5-9cadeceb06f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:11.190694 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08971cdf-4e3d-4518-bce5-9cadeceb06f4 None admin] RBAC: Adding query filter params (group.id=138bdc814b3c4eb9885c7cb5da450386, role.id=3d7c80cf23864f58bf21f893ba7691d6, scope.project.id=e1099d961ac5441c973dfbd04812f431) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:11.191194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08971cdf-4e3d-4518-bce5-9cadeceb06f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:11.193725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08971cdf-4e3d-4518-bce5-9cadeceb06f4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:11.206413 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 227/429] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jul 21 08:56:10 2022] GET /identity/v3/role_assignments?group.id=138bdc814b3c4eb9885c7cb5da450386&scope.project.id=e1099d961ac5441c973dfbd04812f431&role.id=3d7c80cf23864f58bf21f893ba7691d6 => generated 611 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:11.307688 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:11.336114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09f4fc-017a-440c-80e8-f634e74a9763 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:11.391326 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938f394-71d6-4b5d-a2d4-719fbe4c69e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:11.392241 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938f394-71d6-4b5d-a2d4-719fbe4c69e4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:11.392524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938f394-71d6-4b5d-a2d4-719fbe4c69e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:11.392795 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938f394-71d6-4b5d-a2d4-719fbe4c69e4 None admin] PATH_INFO: `/v3/groups/138bdc814b3c4eb9885c7cb5da450386` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:11.393989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938f394-71d6-4b5d-a2d4-719fbe4c69e4 None admin] RBAC: Authorizing `identity:get_group(group_id=138bdc814b3c4eb9885c7cb5da450386)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:11.395454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938f394-71d6-4b5d-a2d4-719fbe4c69e4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:11.397402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 203/430] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jul 21 08:56:11 2022] GET /identity/v3/groups/138bdc814b3c4eb9885c7cb5da450386 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:11.403261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08971cdf-4e3d-4518-bce5-9cadeceb06f4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:11.522675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08971cdf-4e3d-4518-bce5-9cadeceb06f4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:11.559951 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47851a69-116d-41dd-8f53-f84368bbfcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:11.560901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47851a69-116d-41dd-8f53-f84368bbfcb9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:11.561184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47851a69-116d-41dd-8f53-f84368bbfcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:11.561462 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47851a69-116d-41dd-8f53-f84368bbfcb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:11.562305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47851a69-116d-41dd-8f53-f84368bbfcb9 None admin] RBAC: Adding query filter params (group.id=138bdc814b3c4eb9885c7cb5da450386, role.id=5a989bb8c8a1465ca76622ffb504dee5, scope.project.id=e1099d961ac5441c973dfbd04812f431) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:11.562613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47851a69-116d-41dd-8f53-f84368bbfcb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:11.564153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47851a69-116d-41dd-8f53-f84368bbfcb9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:11.570872 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 228/431] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jul 21 08:56:11 2022] GET /identity/v3/role_assignments?group.id=138bdc814b3c4eb9885c7cb5da450386&scope.project.id=e1099d961ac5441c973dfbd04812f431&role.id=5a989bb8c8a1465ca76622ffb504dee5 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:11.624562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937c1ce-dae3-48f2-9a35-e9bf3003d8a2 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:11.624866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937c1ce-dae3-48f2-9a35-e9bf3003d8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:11.625050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937c1ce-dae3-48f2-9a35-e9bf3003d8a2 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:11.626506 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 204/432] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:56:11.634469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57697e-0543-4f97-93e2-e0e9bffea8a2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:11.634791 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57697e-0543-4f97-93e2-e0e9bffea8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:11.635047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57697e-0543-4f97-93e2-e0e9bffea8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:11.641589 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:11.655810 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ef57697e-0543-4f97-93e2-e0e9bffea8a2 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:11.657118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ef57697e-0543-4f97-93e2-e0e9bffea8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:11.677237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60cf21-8508-410f-aae8-aa6336094450 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:11.677595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60cf21-8508-410f-aae8-aa6336094450 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:11.677789 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60cf21-8508-410f-aae8-aa6336094450 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:11.679569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 205/433] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:11.689939 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74847221-2a2a-4360-9d82-88e8e79a5428 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:11.690243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74847221-2a2a-4360-9d82-88e8e79a5428 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:11.690483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74847221-2a2a-4360-9d82-88e8e79a5428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:11.713747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-74847221-2a2a-4360-9d82-88e8e79a5428 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:11.714989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-74847221-2a2a-4360-9d82-88e8e79a5428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:11.779623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 229/434] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:11.807835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57697e-0543-4f97-93e2-e0e9bffea8a2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:11.816057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 206/435] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:56:11.845244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74847221-2a2a-4360-9d82-88e8e79a5428 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:11.931778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57697e-0543-4f97-93e2-e0e9bffea8a2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:11.959819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:11.987842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74847221-2a2a-4360-9d82-88e8e79a5428 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:11.993955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a53f2-04c9-499d-aa92-f92eb292439a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:11.994770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a53f2-04c9-499d-aa92-f92eb292439a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:11.995059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a53f2-04c9-499d-aa92-f92eb292439a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:11.995468 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a53f2-04c9-499d-aa92-f92eb292439a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:12.003041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:12.015049 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a53f2-04c9-499d-aa92-f92eb292439a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-743a53f2-04c9-499d-aa92-f92eb292439a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:12.024761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:12.025870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:12.026722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:12.026722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:12.026722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:12.026722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:12.026722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:12.026722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:12.026722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:12.026722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:12.026722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:12.026722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:12.026722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:56:12.026722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:12.026722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 230/436] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:12.032267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a53f2-04c9-499d-aa92-f92eb292439a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:12.043149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85a2d8-232f-463f-823c-a489af706eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:12.044134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85a2d8-232f-463f-823c-a489af706eaf None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:12.044423 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85a2d8-232f-463f-823c-a489af706eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:12.044691 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85a2d8-232f-463f-823c-a489af706eaf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:12.066558 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85a2d8-232f-463f-823c-a489af706eaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-4f85a2d8-232f-463f-823c-a489af706eaf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:12.076849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:12.078226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:12.079030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:12.079030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:12.079030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:12.079030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:12.079030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:12.079030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:12.079030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:12.079030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:12.079030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:12.079030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:12.079030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:56:12.079030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:12.079030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 207/437] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:56:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:12.083807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85a2d8-232f-463f-823c-a489af706eaf None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:12.181001 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a53f2-04c9-499d-aa92-f92eb292439a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:12.222983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85a2d8-232f-463f-823c-a489af706eaf None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:12.229384 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bc4bc-880e-467d-9fd8-19c55bf30f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:12.230098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bc4bc-880e-467d-9fd8-19c55bf30f3c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:12.230386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bc4bc-880e-467d-9fd8-19c55bf30f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:12.230664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bc4bc-880e-467d-9fd8-19c55bf30f3c None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:12.231530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bc4bc-880e-467d-9fd8-19c55bf30f3c None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:12.232522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bc4bc-880e-467d-9fd8-19c55bf30f3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:12.233969 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bc4bc-880e-467d-9fd8-19c55bf30f3c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:12.244042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 231/438] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:56:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:12.250353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bc4bc-880e-467d-9fd8-19c55bf30f3c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:12.267943 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be0f6f-2ca1-4b56-b975-d552a484dc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:12.269159 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be0f6f-2ca1-4b56-b975-d552a484dc2b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:12.269605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be0f6f-2ca1-4b56-b975-d552a484dc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:12.270014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be0f6f-2ca1-4b56-b975-d552a484dc2b None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:12.271089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be0f6f-2ca1-4b56-b975-d552a484dc2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:12.271823 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be0f6f-2ca1-4b56-b975-d552a484dc2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:12.273340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be0f6f-2ca1-4b56-b975-d552a484dc2b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:12.286429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 208/439] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:12.292731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be0f6f-2ca1-4b56-b975-d552a484dc2b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:12.296090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:12.334457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:12.387603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bc4bc-880e-467d-9fd8-19c55bf30f3c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:12.414757 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be0f6f-2ca1-4b56-b975-d552a484dc2b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:12.424330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e5066-d71b-4f75-9d25-b20d3becf079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:12.424991 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e5066-d71b-4f75-9d25-b20d3becf079 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:12.425277 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e5066-d71b-4f75-9d25-b20d3becf079 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:12.425916 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e5066-d71b-4f75-9d25-b20d3becf079 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:12.454235 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd5297-ff45-46bc-919d-b07b1965aa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:12.454994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd5297-ff45-46bc-919d-b07b1965aa8d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:12.455275 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd5297-ff45-46bc-919d-b07b1965aa8d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:12.455551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd5297-ff45-46bc-919d-b07b1965aa8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:12.498524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e5066-d71b-4f75-9d25-b20d3becf079 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:12.500009 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e5066-d71b-4f75-9d25-b20d3becf079 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:12.536913 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd5297-ff45-46bc-919d-b07b1965aa8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:12.538434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd5297-ff45-46bc-919d-b07b1965aa8d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:12.540369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:12.557154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:12.604034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 232/440] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:12.608955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30655674-8c4b-4a97-83b7-700864620a5e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:12.609188 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30655674-8c4b-4a97-83b7-700864620a5e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:12.609449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30655674-8c4b-4a97-83b7-700864620a5e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:12.610895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 233/441] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:12.611598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30655674-8c4b-4a97-83b7-700864620a5e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:12.647990 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 209/442] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:12.649328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829423d-a482-43d3-99c6-282da04bb056 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:12.649589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829423d-a482-43d3-99c6-282da04bb056 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:12.649867 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829423d-a482-43d3-99c6-282da04bb056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:12.670167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6829423d-a482-43d3-99c6-282da04bb056 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:12.671427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6829423d-a482-43d3-99c6-282da04bb056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:12.723930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30655674-8c4b-4a97-83b7-700864620a5e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:12.749362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 210/443] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:12.750432 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829423d-a482-43d3-99c6-282da04bb056 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:12.760630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68179b08-54b1-497a-b178-f3a9fff73d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:12.761569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68179b08-54b1-497a-b178-f3a9fff73d7b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:12.761779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68179b08-54b1-497a-b178-f3a9fff73d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:12.762038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68179b08-54b1-497a-b178-f3a9fff73d7b None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:12.764772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68179b08-54b1-497a-b178-f3a9fff73d7b None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:12.766230 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68179b08-54b1-497a-b178-f3a9fff73d7b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:12.771987 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 234/444] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:56:12 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:12.773752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68179b08-54b1-497a-b178-f3a9fff73d7b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:12.793472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:12.802013 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:12.870714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829423d-a482-43d3-99c6-282da04bb056 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:12.886596 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68179b08-54b1-497a-b178-f3a9fff73d7b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:12.909561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350ad97-0072-4a78-a828-a8b7d0569fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:12.910427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350ad97-0072-4a78-a828-a8b7d0569fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:12.910710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350ad97-0072-4a78-a828-a8b7d0569fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:12.910981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350ad97-0072-4a78-a828-a8b7d0569fb4 None admin] PATH_INFO: `/v3/projects/551f963b02f4411395171476c9cc2d68` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:12.912211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350ad97-0072-4a78-a828-a8b7d0569fb4 None admin] RBAC: Authorizing `identity:get_project(project_id=551f963b02f4411395171476c9cc2d68)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:12.913681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350ad97-0072-4a78-a828-a8b7d0569fb4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:12.915394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 211/445] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:56:12 2022] GET /identity/v3/projects/551f963b02f4411395171476c9cc2d68 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:12.916195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350ad97-0072-4a78-a828-a8b7d0569fb4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:12.932049 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d5105-e4d0-423b-b655-b40d254b323e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:12.932923 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d5105-e4d0-423b-b655-b40d254b323e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:12.933213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d5105-e4d0-423b-b655-b40d254b323e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:12.933493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d5105-e4d0-423b-b655-b40d254b323e None admin] PATH_INFO: `/v3/groups/138bdc814b3c4eb9885c7cb5da450386` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:12.935570 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d5105-e4d0-423b-b655-b40d254b323e None admin] RBAC: Authorizing `identity:get_group(group_id=138bdc814b3c4eb9885c7cb5da450386)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:12.937194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d5105-e4d0-423b-b655-b40d254b323e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:12.938860 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 235/446] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jul 21 08:56:12 2022] GET /identity/v3/groups/138bdc814b3c4eb9885c7cb5da450386 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:12.939632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d5105-e4d0-423b-b655-b40d254b323e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:13.025047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:13.028498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350ad97-0072-4a78-a828-a8b7d0569fb4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:13.038668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:13.049646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d5105-e4d0-423b-b655-b40d254b323e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:13.065196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277862e-340f-4cc7-916a-bc20de95d3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:13.065987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277862e-340f-4cc7-916a-bc20de95d3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:13.066272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277862e-340f-4cc7-916a-bc20de95d3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:13.066553 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277862e-340f-4cc7-916a-bc20de95d3b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:13.077024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277862e-340f-4cc7-916a-bc20de95d3b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:13.083977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76200e26-a3ba-472e-ba56-b5d63f9df4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-4277862e-340f-4cc7-916a-bc20de95d3b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:13.084673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:13.085666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:13.086440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:13.086440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:13.086440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:13.086440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:13.086440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:13.086440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:13.086440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:13.086440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:13.086440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:13.086440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:13.086440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:13.086440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:13.087051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76200e26-a3ba-472e-ba56-b5d63f9df4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:13.087051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76200e26-a3ba-472e-ba56-b5d63f9df4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:13.087051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76200e26-a3ba-472e-ba56-b5d63f9df4e7 None admin] PATH_INFO: `/v3/groups/138bdc814b3c4eb9885c7cb5da450386` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:13.087051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76200e26-a3ba-472e-ba56-b5d63f9df4e7 None admin] RBAC: Authorizing `identity:get_group(group_id=138bdc814b3c4eb9885c7cb5da450386)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:13.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 212/447] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:56:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:13.089342 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76200e26-a3ba-472e-ba56-b5d63f9df4e7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:13.089440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277862e-340f-4cc7-916a-bc20de95d3b3 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:13.089888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 236/448] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jul 21 08:56:12 2022] GET /identity/v3/groups/138bdc814b3c4eb9885c7cb5da450386 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:13.095742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76200e26-a3ba-472e-ba56-b5d63f9df4e7 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:13.198979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277862e-340f-4cc7-916a-bc20de95d3b3 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:13.215058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76200e26-a3ba-472e-ba56-b5d63f9df4e7 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:13.239181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e30a38-056f-4c8c-bd59-6bb4daa54a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:13.239844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e30a38-056f-4c8c-bd59-6bb4daa54a08 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:13.240177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e30a38-056f-4c8c-bd59-6bb4daa54a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:13.240459 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e30a38-056f-4c8c-bd59-6bb4daa54a08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:13.241301 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e30a38-056f-4c8c-bd59-6bb4daa54a08 None admin] RBAC: Adding query filter params (group.id=138bdc814b3c4eb9885c7cb5da450386, role.id=3d7c80cf23864f58bf21f893ba7691d6, scope.project.id=551f963b02f4411395171476c9cc2d68) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:13.241618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e30a38-056f-4c8c-bd59-6bb4daa54a08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:13.243086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e30a38-056f-4c8c-bd59-6bb4daa54a08 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:13.250298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 213/449] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jul 21 08:56:13 2022] GET /identity/v3/role_assignments?group.id=138bdc814b3c4eb9885c7cb5da450386&scope.project.id=551f963b02f4411395171476c9cc2d68&role.id=3d7c80cf23864f58bf21f893ba7691d6 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:13.251334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e30a38-056f-4c8c-bd59-6bb4daa54a08 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:13.285672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:13.287241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd43ffb-c7a8-4254-82be-106af7ea7895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:13.287968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd43ffb-c7a8-4254-82be-106af7ea7895 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:13.288616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd43ffb-c7a8-4254-82be-106af7ea7895 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:13.288616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd43ffb-c7a8-4254-82be-106af7ea7895 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:13.289297 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd43ffb-c7a8-4254-82be-106af7ea7895 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:13.289933 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd43ffb-c7a8-4254-82be-106af7ea7895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:13.291351 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd43ffb-c7a8-4254-82be-106af7ea7895 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:13.312827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 237/450] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:13.318687 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd43ffb-c7a8-4254-82be-106af7ea7895 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:13.357108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:13.382749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e30a38-056f-4c8c-bd59-6bb4daa54a08 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:13.432629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34761a19-1106-42b2-99cb-645ba9cffe5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:13.433298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34761a19-1106-42b2-99cb-645ba9cffe5f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:13.433584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34761a19-1106-42b2-99cb-645ba9cffe5f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:13.433859 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34761a19-1106-42b2-99cb-645ba9cffe5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:13.436424 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34761a19-1106-42b2-99cb-645ba9cffe5f None admin] RBAC: Adding query filter params (group.id=138bdc814b3c4eb9885c7cb5da450386, role.id=5a989bb8c8a1465ca76622ffb504dee5, scope.project.id=551f963b02f4411395171476c9cc2d68) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:13.436746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34761a19-1106-42b2-99cb-645ba9cffe5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:13.438921 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34761a19-1106-42b2-99cb-645ba9cffe5f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:13.446438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 214/451] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jul 21 08:56:13 2022] GET /identity/v3/role_assignments?group.id=138bdc814b3c4eb9885c7cb5da450386&scope.project.id=551f963b02f4411395171476c9cc2d68&role.id=5a989bb8c8a1465ca76622ffb504dee5 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:13.467352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd43ffb-c7a8-4254-82be-106af7ea7895 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:13.510815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cda20b-e81a-4fce-835b-de77cd529a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:13.511955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cda20b-e81a-4fce-835b-de77cd529a2e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:13.512247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cda20b-e81a-4fce-835b-de77cd529a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:13.513276 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cda20b-e81a-4fce-835b-de77cd529a2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:13.611843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cda20b-e81a-4fce-835b-de77cd529a2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:13.613356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cda20b-e81a-4fce-835b-de77cd529a2e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:13.642422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:13.725954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 238/452] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:13.732647 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34761a19-1106-42b2-99cb-645ba9cffe5f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:13.828346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:13.842231 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34761a19-1106-42b2-99cb-645ba9cffe5f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:13.875854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ec60e-69e8-4b7c-8164-8039ea452611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:13.876662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ec60e-69e8-4b7c-8164-8039ea452611 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:13.876947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ec60e-69e8-4b7c-8164-8039ea452611 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:13.877224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ec60e-69e8-4b7c-8164-8039ea452611 None admin] PATH_INFO: `/v3/projects/a27dd5dc3697487bbb9b172396ff08c2` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:13.878402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ec60e-69e8-4b7c-8164-8039ea452611 None admin] RBAC: Authorizing `identity:get_project(project_id=a27dd5dc3697487bbb9b172396ff08c2)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:13.879856 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ec60e-69e8-4b7c-8164-8039ea452611 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:13.881639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 215/453] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:56:13 2022] GET /identity/v3/projects/a27dd5dc3697487bbb9b172396ff08c2 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:13.887819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cda20b-e81a-4fce-835b-de77cd529a2e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:13.989132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cda20b-e81a-4fce-835b-de77cd529a2e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:14.010826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:14.023265 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83874cf4-8279-43e1-b7a5-11d402b46c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:14.024145 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83874cf4-8279-43e1-b7a5-11d402b46c27 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:14.024432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83874cf4-8279-43e1-b7a5-11d402b46c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:14.024707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83874cf4-8279-43e1-b7a5-11d402b46c27 None admin] PATH_INFO: `/v3/groups/2cda83ee8d9e485c8e511e52510ff97f` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:14.025858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83874cf4-8279-43e1-b7a5-11d402b46c27 None admin] RBAC: Authorizing `identity:get_group(group_id=2cda83ee8d9e485c8e511e52510ff97f)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:14.027311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83874cf4-8279-43e1-b7a5-11d402b46c27 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:14.029571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 239/454] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jul 21 08:56:13 2022] GET /identity/v3/groups/2cda83ee8d9e485c8e511e52510ff97f => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:14.035419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ec60e-69e8-4b7c-8164-8039ea452611 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:14.133335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ec60e-69e8-4b7c-8164-8039ea452611 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:14.166629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ab188-ae7f-4f0e-b28d-d577bcb3b065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:14.167522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ab188-ae7f-4f0e-b28d-d577bcb3b065 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:14.167802 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ab188-ae7f-4f0e-b28d-d577bcb3b065 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:14.168101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ab188-ae7f-4f0e-b28d-d577bcb3b065 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:14.168959 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ab188-ae7f-4f0e-b28d-d577bcb3b065 None admin] RBAC: Adding query filter params (group.id=2cda83ee8d9e485c8e511e52510ff97f, role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=a27dd5dc3697487bbb9b172396ff08c2) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:14.169268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ab188-ae7f-4f0e-b28d-d577bcb3b065 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:14.170705 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ab188-ae7f-4f0e-b28d-d577bcb3b065 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:14.176367 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 216/455] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jul 21 08:56:14 2022] GET /identity/v3/role_assignments?group.id=2cda83ee8d9e485c8e511e52510ff97f&scope.project.id=a27dd5dc3697487bbb9b172396ff08c2&role.id=000e942517f2469ba36d2e65978bc314 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:16.719425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c434dc-5292-4d86-adde-efa5e8f98d97 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:16.720249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c434dc-5292-4d86-adde-efa5e8f98d97 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:16.720696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c434dc-5292-4d86-adde-efa5e8f98d97 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:16.722479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 240/456] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:16.728170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd87bf-e187-4d3f-b339-947b82640799 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:16.729004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd87bf-e187-4d3f-b339-947b82640799 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:16.729455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd87bf-e187-4d3f-b339-947b82640799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:16.750394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b0cd87bf-e187-4d3f-b339-947b82640799 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:16.751610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b0cd87bf-e187-4d3f-b339-947b82640799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:16.777141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:16.803697 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ca499-901e-442f-8fd9-d8b6533b8511 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:16.804229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ca499-901e-442f-8fd9-d8b6533b8511 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:16.804650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ca499-901e-442f-8fd9-d8b6533b8511 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:16.807559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 241/457] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:16.812257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f5307-9961-4947-ad10-066155ec2674 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:16.812838 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f5307-9961-4947-ad10-066155ec2674 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:16.813312 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f5307-9961-4947-ad10-066155ec2674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:16.842650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 217/458] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:16.848996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-581f5307-9961-4947-ad10-066155ec2674 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:16.850256 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-581f5307-9961-4947-ad10-066155ec2674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:16.894763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd87bf-e187-4d3f-b339-947b82640799 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:16.930380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 242/459] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:16.977410 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f5307-9961-4947-ad10-066155ec2674 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:17.097685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f5307-9961-4947-ad10-066155ec2674 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:17.125935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd87bf-e187-4d3f-b339-947b82640799 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:17.132381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:17.143050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ede6c-66d5-4aec-9e53-ae6f5310a9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:17.144314 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ede6c-66d5-4aec-9e53-ae6f5310a9cc None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:17.144823 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ede6c-66d5-4aec-9e53-ae6f5310a9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:17.145303 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ede6c-66d5-4aec-9e53-ae6f5310a9cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:17.170632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ede6c-66d5-4aec-9e53-ae6f5310a9cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:17.172515 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ede6c-66d5-4aec-9e53-ae6f5310a9cc None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:17.202721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-b68ede6c-66d5-4aec-9e53-ae6f5310a9cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:17.202721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:17.202721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:17.202721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:17.202721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:17.202721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:17.202721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:17.202721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:17.202721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:17.202721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:17.202721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:17.202721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:17.202721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:17.202721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:17.202721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:17.203653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:17.204554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:17.204554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:17.204554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:17.204554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:17.209184 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bb86f-61eb-4040-84fd-a22efcda9727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:17.210256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bb86f-61eb-4040-84fd-a22efcda9727 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:17.210746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bb86f-61eb-4040-84fd-a22efcda9727 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:17.211224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bb86f-61eb-4040-84fd-a22efcda9727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:17.214760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 243/460] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:17.221075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ede6c-66d5-4aec-9e53-ae6f5310a9cc None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:17.227727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bb86f-61eb-4040-84fd-a22efcda9727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:17.231147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bb86f-61eb-4040-84fd-a22efcda9727 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:17.265407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-d38bb86f-61eb-4040-84fd-a22efcda9727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:17.265407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:17.265407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:17.265407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:17.265407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:17.265407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:17.265407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:17.265407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:17.265407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:17.265407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:17.265407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:17.265407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:17.265407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:17.265407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:17.265407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:17.266325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:17.267169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:17.267169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:17.267169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:17.267169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:17.270573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 218/461] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:17.272121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aec70b-a838-4d29-9d9c-aa8593479617 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:17.272638 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aec70b-a838-4d29-9d9c-aa8593479617 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:17.273111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aec70b-a838-4d29-9d9c-aa8593479617 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:17.275547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 219/462] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:17.276538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aec70b-a838-4d29-9d9c-aa8593479617 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:17.279709 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:17.345469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ede6c-66d5-4aec-9e53-ae6f5310a9cc None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:17.383625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1232b-afec-4ed8-84ff-a06ef421945c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:17.384321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1232b-afec-4ed8-84ff-a06ef421945c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:17.384796 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1232b-afec-4ed8-84ff-a06ef421945c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:17.385211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1232b-afec-4ed8-84ff-a06ef421945c None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:17.386088 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1232b-afec-4ed8-84ff-a06ef421945c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:17.386535 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1232b-afec-4ed8-84ff-a06ef421945c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:17.388786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1232b-afec-4ed8-84ff-a06ef421945c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:17.406701 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 244/463] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:17.408386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662c5fc-5100-4efe-a678-fa40b08338bd None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:17.408822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662c5fc-5100-4efe-a678-fa40b08338bd None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:17.409242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662c5fc-5100-4efe-a678-fa40b08338bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:17.427783 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aec70b-a838-4d29-9d9c-aa8593479617 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:17.428012 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d662c5fc-5100-4efe-a678-fa40b08338bd None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:17.429351 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d662c5fc-5100-4efe-a678-fa40b08338bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:17.457347 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:17.475605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1db73b-ca8d-49ba-9493-27c1fef10ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:17.476458 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1db73b-ca8d-49ba-9493-27c1fef10ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:17.477326 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1db73b-ca8d-49ba-9493-27c1fef10ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:17.477714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1db73b-ca8d-49ba-9493-27c1fef10ff9 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:17.478643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1db73b-ca8d-49ba-9493-27c1fef10ff9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:17.479008 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1db73b-ca8d-49ba-9493-27c1fef10ff9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:17.480487 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1db73b-ca8d-49ba-9493-27c1fef10ff9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:17.493107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 220/464] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:17.494235 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1db73b-ca8d-49ba-9493-27c1fef10ff9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:17.508422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 245/465] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:17.510011 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7424af-3ad6-44b4-b858-ca3835ac6fad None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:17.510473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7424af-3ad6-44b4-b858-ca3835ac6fad None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:17.510916 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7424af-3ad6-44b4-b858-ca3835ac6fad None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:17.512397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 246/466] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:17.513878 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7424af-3ad6-44b4-b858-ca3835ac6fad None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:17.569406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:17.623739 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1db73b-ca8d-49ba-9493-27c1fef10ff9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:17.662650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7424af-3ad6-44b4-b858-ca3835ac6fad None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:17.673607 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939da2fc-90e2-4dc7-956f-6dbfbf6ae998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:17.674734 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939da2fc-90e2-4dc7-956f-6dbfbf6ae998 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:17.675540 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939da2fc-90e2-4dc7-956f-6dbfbf6ae998 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:17.676022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939da2fc-90e2-4dc7-956f-6dbfbf6ae998 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:17.677251 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939da2fc-90e2-4dc7-956f-6dbfbf6ae998 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:17.679348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939da2fc-90e2-4dc7-956f-6dbfbf6ae998 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:17.704111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd2878-9317-4aac-bdcf-b187446dd2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:17.704523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 221/467] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:56:17 2022] POST /identity/v3/users => generated 248 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:56:17.704796 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd2878-9317-4aac-bdcf-b187446dd2a0 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:17.705081 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd2878-9317-4aac-bdcf-b187446dd2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:17.705356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd2878-9317-4aac-bdcf-b187446dd2a0 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:17.706082 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd2878-9317-4aac-bdcf-b187446dd2a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:17.707592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd2878-9317-4aac-bdcf-b187446dd2a0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:17.714682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31900a4-711e-4918-bc17-4070d9cb20e4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:17.714682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31900a4-711e-4918-bc17-4070d9cb20e4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:17.714682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31900a4-711e-4918-bc17-4070d9cb20e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:17.739295 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 247/468] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:56:17 2022] POST /identity/v3/users => generated 246 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:56:17.741674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd2878-9317-4aac-bdcf-b187446dd2a0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:17.752240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f31900a4-711e-4918-bc17-4070d9cb20e4 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:17.755533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f31900a4-711e-4918-bc17-4070d9cb20e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:17.771267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:17.870403 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 222/469] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:17.913260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31900a4-711e-4918-bc17-4070d9cb20e4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:17.925905 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd2878-9317-4aac-bdcf-b187446dd2a0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:17.997560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e3a8d-fa49-402c-88fd-37aa73f03d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:17.998310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e3a8d-fa49-402c-88fd-37aa73f03d45 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:17.998599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e3a8d-fa49-402c-88fd-37aa73f03d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:17.998873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e3a8d-fa49-402c-88fd-37aa73f03d45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:18.012039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e3a8d-fa49-402c-88fd-37aa73f03d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:18.013722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e3a8d-fa49-402c-88fd-37aa73f03d45 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:18.021796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:18.035417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-dc3e3a8d-fa49-402c-88fd-37aa73f03d45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:18.035417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:18.035417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:18.035417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:18.035417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:18.035417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:18.035417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:18.035417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:18.035417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:18.035417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:18.035417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:18.035417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:18.035417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:18.035417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:18.035417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:18.036416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:18.037255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:18.037255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:18.037255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:18.037255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:18.037255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 248/470] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:18.042982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e3a8d-fa49-402c-88fd-37aa73f03d45 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:18.110292 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31900a4-711e-4918-bc17-4070d9cb20e4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:18.152838 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6a38f-2e8d-4623-af07-911be083caf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:18.154377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6a38f-2e8d-4623-af07-911be083caf9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:18.154668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6a38f-2e8d-4623-af07-911be083caf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:18.154941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6a38f-2e8d-4623-af07-911be083caf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:18.155703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:18.169721 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6a38f-2e8d-4623-af07-911be083caf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:18.172286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6a38f-2e8d-4623-af07-911be083caf9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:18.197111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e3a8d-fa49-402c-88fd-37aa73f03d45 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:18.199233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-4bf6a38f-2e8d-4623-af07-911be083caf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:18.199233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:18.199233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:18.199233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:18.199233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:18.199233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:18.199233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:18.199233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:18.199233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:18.199233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:18.199233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:18.199233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:18.199233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:18.199233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:18.199233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:18.200137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:18.201038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:18.201038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:18.201038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:18.201038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:18.202530 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 223/471] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:18.207386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5722b33-86a4-480e-be67-aba492d85f51 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:18.207386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5722b33-86a4-480e-be67-aba492d85f51 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:18.207622 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5722b33-86a4-480e-be67-aba492d85f51 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:18.209624 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 224/472] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:18.210107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5722b33-86a4-480e-be67-aba492d85f51 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:18.250100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9baef-f707-4540-bce4-d06aa69fbae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:18.251141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9baef-f707-4540-bce4-d06aa69fbae2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:18.251956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9baef-f707-4540-bce4-d06aa69fbae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:18.253389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9baef-f707-4540-bce4-d06aa69fbae2 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:18.254437 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9baef-f707-4540-bce4-d06aa69fbae2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:18.254955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9baef-f707-4540-bce4-d06aa69fbae2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:18.256603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9baef-f707-4540-bce4-d06aa69fbae2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:18.273432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 249/473] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:18.275068 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770423b-3951-4b66-82a2-d5ac07a92627 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:18.275620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770423b-3951-4b66-82a2-d5ac07a92627 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:18.276139 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770423b-3951-4b66-82a2-d5ac07a92627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:18.296115 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d770423b-3951-4b66-82a2-d5ac07a92627 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:18.297361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d770423b-3951-4b66-82a2-d5ac07a92627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:18.354106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:18.359084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5722b33-86a4-480e-be67-aba492d85f51 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:18.386611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 250/474] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:18.388958 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770423b-3951-4b66-82a2-d5ac07a92627 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:18.410465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6b4f2-1d50-42bf-a1d1-709d1e672fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:18.411195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6b4f2-1d50-42bf-a1d1-709d1e672fad None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:18.411529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6b4f2-1d50-42bf-a1d1-709d1e672fad None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:18.411986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6b4f2-1d50-42bf-a1d1-709d1e672fad None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:18.412748 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6b4f2-1d50-42bf-a1d1-709d1e672fad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:18.413059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6b4f2-1d50-42bf-a1d1-709d1e672fad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:18.414892 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6b4f2-1d50-42bf-a1d1-709d1e672fad None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:18.432392 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 225/475] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:18.433343 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6b4f2-1d50-42bf-a1d1-709d1e672fad None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:18.524995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770423b-3951-4b66-82a2-d5ac07a92627 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:18.551138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6b4f2-1d50-42bf-a1d1-709d1e672fad None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:18.564330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:18.576827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6fd8-ec73-44ea-ac64-16d5199ca128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:18.577363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6fd8-ec73-44ea-ac64-16d5199ca128 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:18.577594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6fd8-ec73-44ea-ac64-16d5199ca128 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:18.577863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba6fd8-ec73-44ea-ac64-16d5199ca128 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:18.579433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba6fd8-ec73-44ea-ac64-16d5199ca128 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:18.581164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba6fd8-ec73-44ea-ac64-16d5199ca128 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:18.599029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccbed6-b407-41b7-8e9d-c7baad29813a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:18.600335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccbed6-b407-41b7-8e9d-c7baad29813a None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:18.600840 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccbed6-b407-41b7-8e9d-c7baad29813a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:18.601465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccbed6-b407-41b7-8e9d-c7baad29813a None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:18.602459 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccbed6-b407-41b7-8e9d-c7baad29813a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:18.604128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccbed6-b407-41b7-8e9d-c7baad29813a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:18.612023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 251/476] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:56:18 2022] POST /identity/v3/users => generated 248 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:56:18.613415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6fd8-ec73-44ea-ac64-16d5199ca128 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:18.620723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.sql.core [None req-e7ccbed6-b407-41b7-8e9d-c7baad29813a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 08:56:18.620723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 08:56:18.620723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [parameters: {'id': '435a9bea446f4e00b10242e07266b901', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 08:56:18.620723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80199) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 08:56:18.654355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-e7ccbed6-b407-41b7-8e9d-c7baad29813a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 08:56:18.654355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:18.654355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 08:56:18.654355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 08:56:18.654355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 08:56:18.654355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 08:56:18.654355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 08:56:18.654355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 08:56:18.654355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 08:56:18.654355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application conn.query(q) Jul 21 08:56:18.654355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 08:56:18.654355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 08:56:18.654355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 08:56:18.654355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application result.read() Jul 21 08:56:18.654355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return ref.to_dict() Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 08:56:18.655362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application next(self.gen) Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application yield resource Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application next(self.gen) Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise self.value Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application session.commit() Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 08:56:18.656479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self.session.flush() Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application compat.raise_( Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 08:56:18.657399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 08:56:18.658334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application conn.query(q) Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application result.read() Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 08:56:18.659216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [parameters: {'id': '435a9bea446f4e00b10242e07266b901', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:18.662258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 08:56:18.663418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:18.667057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 226/477] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Jul 21 08:56:18 2022] POST /identity/v3/roles => generated 144 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:18.673170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccbed6-b407-41b7-8e9d-c7baad29813a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:18.740208 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:18.846853 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba6fd8-ec73-44ea-ac64-16d5199ca128 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:18.870572 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:18.883520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccbed6-b407-41b7-8e9d-c7baad29813a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:18.897116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c122096-649c-4f82-83df-85b91ea28dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:18.897806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c122096-649c-4f82-83df-85b91ea28dca None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:18.898099 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c122096-649c-4f82-83df-85b91ea28dca None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:18.898410 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c122096-649c-4f82-83df-85b91ea28dca None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:18.899564 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c122096-649c-4f82-83df-85b91ea28dca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:18.901940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c122096-649c-4f82-83df-85b91ea28dca None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:18.933961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 252/478] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:56:18 2022] POST /identity/v3/users => generated 249 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:56:18.936171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735c189-8cba-4b11-b6c1-e80a36ddca4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:18.937051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735c189-8cba-4b11-b6c1-e80a36ddca4d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:18.937364 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735c189-8cba-4b11-b6c1-e80a36ddca4d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:18.937643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735c189-8cba-4b11-b6c1-e80a36ddca4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:18.956466 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735c189-8cba-4b11-b6c1-e80a36ddca4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-d735c189-8cba-4b11-b6c1-e80a36ddca4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:18.965305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:18.966520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:18.967321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:18.967321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:18.967321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:18.967321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:18.967321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:18.967321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:18.967321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:18.967321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:18.967321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:18.967321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:18.967321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:56:18.967321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:18.967321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 227/479] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:56:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:18.974876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c122096-649c-4f82-83df-85b91ea28dca None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:19.112129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c122096-649c-4f82-83df-85b91ea28dca None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:19.147890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdaf818-eb53-491a-9181-f0d68c3160e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:19.148602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdaf818-eb53-491a-9181-f0d68c3160e4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:19.148881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdaf818-eb53-491a-9181-f0d68c3160e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:19.149152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdaf818-eb53-491a-9181-f0d68c3160e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:19.149912 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdaf818-eb53-491a-9181-f0d68c3160e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:19.150212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdaf818-eb53-491a-9181-f0d68c3160e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:19.151720 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdaf818-eb53-491a-9181-f0d68c3160e4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:19.162425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 253/480] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:20.142383 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507ecf5-80c5-4c33-8474-e2acdcbcb0c6 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:20.142907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507ecf5-80c5-4c33-8474-e2acdcbcb0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:20.142989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507ecf5-80c5-4c33-8474-e2acdcbcb0c6 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:20.144629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 228/481] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:20.149865 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db07bf3-d43a-48d3-bdf6-2f8221486e34 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:20.150296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db07bf3-d43a-48d3-bdf6-2f8221486e34 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:20.150475 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db07bf3-d43a-48d3-bdf6-2f8221486e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:20.167394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4db07bf3-d43a-48d3-bdf6-2f8221486e34 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:20.168882 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4db07bf3-d43a-48d3-bdf6-2f8221486e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:20.210535 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:20.265917 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 254/482] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:20.308822 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507ecf5-80c5-4c33-8474-e2acdcbcb0c6 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:20.420967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7cd06-d15e-431e-9e3e-58ebc1e97340 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:20.421149 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7cd06-d15e-431e-9e3e-58ebc1e97340 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:20.421433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7cd06-d15e-431e-9e3e-58ebc1e97340 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:20.423237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 255/483] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:56:20.428396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ca284-10e8-4af7-ae9c-d63aeb3302d3 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:20.428685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ca284-10e8-4af7-ae9c-d63aeb3302d3 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:20.428966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ca284-10e8-4af7-ae9c-d63aeb3302d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:20.432297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:20.446811 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d52ca284-10e8-4af7-ae9c-d63aeb3302d3 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:20.448324 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d52ca284-10e8-4af7-ae9c-d63aeb3302d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:20.463855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507ecf5-80c5-4c33-8474-e2acdcbcb0c6 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:20.536975 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9519f9-02a5-4f7b-9fc2-69b3810b3a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:20.537903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9519f9-02a5-4f7b-9fc2-69b3810b3a4a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:20.538181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9519f9-02a5-4f7b-9fc2-69b3810b3a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:20.538452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9519f9-02a5-4f7b-9fc2-69b3810b3a4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:20.561599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9519f9-02a5-4f7b-9fc2-69b3810b3a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-9c9519f9-02a5-4f7b-9fc2-69b3810b3a4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:20.578640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:20.579808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:20.580742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:20.580742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:20.580742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:20.580742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:20.580742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:20.580742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:20.580742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:20.580742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:20.580742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:20.580742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:20.580742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:20.580742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:20.582641 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 229/484] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:20.590019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9519f9-02a5-4f7b-9fc2-69b3810b3a4a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:20.597589 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 256/485] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:56:20.638445 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ca284-10e8-4af7-ae9c-d63aeb3302d3 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:20.838385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9519f9-02a5-4f7b-9fc2-69b3810b3a4a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:20.848339 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:20.862414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ca284-10e8-4af7-ae9c-d63aeb3302d3 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:20.902145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36963855-6250-4890-be34-27a54d84354c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:20.902875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36963855-6250-4890-be34-27a54d84354c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:20.903156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36963855-6250-4890-be34-27a54d84354c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:20.904019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36963855-6250-4890-be34-27a54d84354c None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:20.904842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36963855-6250-4890-be34-27a54d84354c None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:20.905196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36963855-6250-4890-be34-27a54d84354c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:20.906588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36963855-6250-4890-be34-27a54d84354c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:20.921814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 230/486] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:20.929047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36963855-6250-4890-be34-27a54d84354c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:20.937819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42d8d0-bfd0-4132-aa45-86615a809634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:20.938684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42d8d0-bfd0-4132-aa45-86615a809634 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:20.938964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42d8d0-bfd0-4132-aa45-86615a809634 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:20.939233 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42d8d0-bfd0-4132-aa45-86615a809634 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:20.943588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:20.964833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42d8d0-bfd0-4132-aa45-86615a809634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-cc42d8d0-bfd0-4132-aa45-86615a809634 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:20.987757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:20.988718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:20.989520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:20.989520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:20.989520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:20.989520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:20.989520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:20.989520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:20.989520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:20.989520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:20.989520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:20.989520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:20.989520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:20.989520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:20.990839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 257/487] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:20.995652 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42d8d0-bfd0-4132-aa45-86615a809634 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:21.127744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36963855-6250-4890-be34-27a54d84354c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:21.171273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3490ba-64ea-48ef-9d72-1fc6e7fcc306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:21.172098 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3490ba-64ea-48ef-9d72-1fc6e7fcc306 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:21.172374 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3490ba-64ea-48ef-9d72-1fc6e7fcc306 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:21.172649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3490ba-64ea-48ef-9d72-1fc6e7fcc306 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:21.201295 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3490ba-64ea-48ef-9d72-1fc6e7fcc306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:21.202858 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3490ba-64ea-48ef-9d72-1fc6e7fcc306 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:21.221224 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42d8d0-bfd0-4132-aa45-86615a809634 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:21.235507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-af3490ba-64ea-48ef-9d72-1fc6e7fcc306 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:21.235507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:21.235507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:21.235507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:21.235507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:21.235507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:21.235507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:21.235507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:21.235507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:21.235507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:21.235507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:21.235507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:21.235507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:21.235507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:21.235507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:21.236471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:21.237821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:21.237821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:21.237821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:21.237821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:21.237821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 231/488] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:21.243776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3490ba-64ea-48ef-9d72-1fc6e7fcc306 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:21.281826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f923419-5906-469b-8c40-ef033ab45a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:21.282557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f923419-5906-469b-8c40-ef033ab45a8d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:21.282956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f923419-5906-469b-8c40-ef033ab45a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:21.283230 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f923419-5906-469b-8c40-ef033ab45a8d None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:21.284125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f923419-5906-469b-8c40-ef033ab45a8d None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:21.284425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f923419-5906-469b-8c40-ef033ab45a8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:21.285809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f923419-5906-469b-8c40-ef033ab45a8d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:21.295973 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 258/489] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:21.297483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff00fd-ea4b-471b-a248-ca49c3fc9865 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:21.297767 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff00fd-ea4b-471b-a248-ca49c3fc9865 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:21.298041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff00fd-ea4b-471b-a248-ca49c3fc9865 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:21.303561 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 259/490] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:21.304552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff00fd-ea4b-471b-a248-ca49c3fc9865 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:21.319981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:21.336297 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:21.380819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3490ba-64ea-48ef-9d72-1fc6e7fcc306 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:21.424579 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c030b9-28ac-4f2f-a661-12a54e97a321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:21.425368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c030b9-28ac-4f2f-a661-12a54e97a321 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:21.425573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c030b9-28ac-4f2f-a661-12a54e97a321 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:21.425855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c030b9-28ac-4f2f-a661-12a54e97a321 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:21.426592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c030b9-28ac-4f2f-a661-12a54e97a321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:21.426897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c030b9-28ac-4f2f-a661-12a54e97a321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:21.428400 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c030b9-28ac-4f2f-a661-12a54e97a321 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:21.433786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff00fd-ea4b-471b-a248-ca49c3fc9865 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:21.446970 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 232/491] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:21.446970 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0cb7b-6b8f-4a21-bab6-6fdc0b24a984 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:21.446970 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0cb7b-6b8f-4a21-bab6-6fdc0b24a984 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:21.446970 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0cb7b-6b8f-4a21-bab6-6fdc0b24a984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:21.464846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4ec0cb7b-6b8f-4a21-bab6-6fdc0b24a984 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:21.466387 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4ec0cb7b-6b8f-4a21-bab6-6fdc0b24a984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:21.483675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3359b6-f20c-433c-aeea-fdd4eba4bc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:21.485539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3359b6-f20c-433c-aeea-fdd4eba4bc8f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:21.486285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3359b6-f20c-433c-aeea-fdd4eba4bc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:21.486780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3359b6-f20c-433c-aeea-fdd4eba4bc8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:21.503116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3359b6-f20c-433c-aeea-fdd4eba4bc8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:21.505480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3359b6-f20c-433c-aeea-fdd4eba4bc8f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:21.530685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-ee3359b6-f20c-433c-aeea-fdd4eba4bc8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:21.530685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:21.530685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:21.530685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:21.530685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:21.530685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:21.530685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:21.530685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:21.530685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:21.530685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:21.530685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:21.530685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:21.530685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:21.530685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:21.530685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:21.531602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:21.532512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:21.532512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:21.532512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:21.532512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:21.538273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 260/492] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:21.538273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3359b6-f20c-433c-aeea-fdd4eba4bc8f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:21.550407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 233/493] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:21.560680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0cb7b-6b8f-4a21-bab6-6fdc0b24a984 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:21.617990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:21.634612 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:21.660712 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3359b6-f20c-433c-aeea-fdd4eba4bc8f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:21.703770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0cb7b-6b8f-4a21-bab6-6fdc0b24a984 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:21.704021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67218e3-f133-4ded-8cf7-9ffbc03aeab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:21.704725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67218e3-f133-4ded-8cf7-9ffbc03aeab3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:21.705012 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67218e3-f133-4ded-8cf7-9ffbc03aeab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:21.705292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67218e3-f133-4ded-8cf7-9ffbc03aeab3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:21.717253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67218e3-f133-4ded-8cf7-9ffbc03aeab3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:21.719340 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67218e3-f133-4ded-8cf7-9ffbc03aeab3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-a67218e3-f133-4ded-8cf7-9ffbc03aeab3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:21.737442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:21.738497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:21.744664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:21.744664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:21.744664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:21.744664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:21.744664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:21.744664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:21.744664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:21.744664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:21.744664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:56:21.744664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:21.744664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 261/494] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jul 21 08:56:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:21.744664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67218e3-f133-4ded-8cf7-9ffbc03aeab3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:21.761472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44662b-e09b-4b8d-be9c-8c772a8c4614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:21.762064 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44662b-e09b-4b8d-be9c-8c772a8c4614 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:21.762354 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44662b-e09b-4b8d-be9c-8c772a8c4614 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:21.762628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44662b-e09b-4b8d-be9c-8c772a8c4614 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:21.765101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44662b-e09b-4b8d-be9c-8c772a8c4614 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:21.765429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44662b-e09b-4b8d-be9c-8c772a8c4614 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:21.766900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44662b-e09b-4b8d-be9c-8c772a8c4614 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:21.781659 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 234/495] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:21.783017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0583721-2370-4951-a3c7-d1d6b5c57dcf None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:21.783307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0583721-2370-4951-a3c7-d1d6b5c57dcf None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:21.783582 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0583721-2370-4951-a3c7-d1d6b5c57dcf None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:21.787971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 235/496] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:21.787971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0583721-2370-4951-a3c7-d1d6b5c57dcf None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:21.866633 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67218e3-f133-4ded-8cf7-9ffbc03aeab3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:21.894857 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:21.906084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296e4f8-e314-4ba5-8211-ee8e6db62727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:21.906886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296e4f8-e314-4ba5-8211-ee8e6db62727 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:21.907168 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296e4f8-e314-4ba5-8211-ee8e6db62727 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:21.907441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296e4f8-e314-4ba5-8211-ee8e6db62727 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:21.908223 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0583721-2370-4951-a3c7-d1d6b5c57dcf None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:21.917458 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:21.918151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296e4f8-e314-4ba5-8211-ee8e6db62727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-1296e4f8-e314-4ba5-8211-ee8e6db62727 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:21.927121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:21.928290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:21.929431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:21.929431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:21.929431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:21.929431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:21.929431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:21.929431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:21.929431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:21.929431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:21.929431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:21.929431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:21.929431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:21.929431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:21.929431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 262/497] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:21.929431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296e4f8-e314-4ba5-8211-ee8e6db62727 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:21.946059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2ac10-839a-44cb-a577-8cf2abe01e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:21.946833 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2ac10-839a-44cb-a577-8cf2abe01e6f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:21.947120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2ac10-839a-44cb-a577-8cf2abe01e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:21.947395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2ac10-839a-44cb-a577-8cf2abe01e6f None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:21.948280 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2ac10-839a-44cb-a577-8cf2abe01e6f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:21.948591 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2ac10-839a-44cb-a577-8cf2abe01e6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:21.950065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2ac10-839a-44cb-a577-8cf2abe01e6f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:21.987588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 236/498] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jul 21 08:56:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:21.990573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6334a2-73e3-4c36-bcce-15ac4a18774c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:21.990870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6334a2-73e3-4c36-bcce-15ac4a18774c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:21.991152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6334a2-73e3-4c36-bcce-15ac4a18774c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:22.007506 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4e6334a2-73e3-4c36-bcce-15ac4a18774c None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:22.008734 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4e6334a2-73e3-4c36-bcce-15ac4a18774c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:22.036525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296e4f8-e314-4ba5-8211-ee8e6db62727 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:22.070042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f6df4-8544-4441-a81e-b33608c3d957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:22.070840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f6df4-8544-4441-a81e-b33608c3d957 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:22.071124 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f6df4-8544-4441-a81e-b33608c3d957 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:22.071396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f6df4-8544-4441-a81e-b33608c3d957 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:22.078593 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 237/499] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:22.079948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597c62c-bbca-455a-99b8-1fad949400e6 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:22.080248 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597c62c-bbca-455a-99b8-1fad949400e6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:22.080527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597c62c-bbca-455a-99b8-1fad949400e6 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:22.081821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 238/500] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:22.082497 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597c62c-bbca-455a-99b8-1fad949400e6 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:22.089192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f6df4-8544-4441-a81e-b33608c3d957 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:22.090694 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f6df4-8544-4441-a81e-b33608c3d957 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-d85f6df4-8544-4441-a81e-b33608c3d957 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:22.106264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:22.107536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:22.108474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:22.108474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:22.108474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:22.108474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:22.108474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:22.108474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:22.108474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:22.108474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:22.108474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 08:56:22.108474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:22.108474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 263/501] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:22.109045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f6df4-8544-4441-a81e-b33608c3d957 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:22.159222 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:22.193615 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:22.203256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597c62c-bbca-455a-99b8-1fad949400e6 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:22.210311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f6df4-8544-4441-a81e-b33608c3d957 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:22.238711 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51fdaf-144c-4610-ade4-8f2201d68a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:22.239377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51fdaf-144c-4610-ade4-8f2201d68a2c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:22.239666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51fdaf-144c-4610-ade4-8f2201d68a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:22.239973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51fdaf-144c-4610-ade4-8f2201d68a2c None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:22.240723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51fdaf-144c-4610-ade4-8f2201d68a2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:22.241034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51fdaf-144c-4610-ade4-8f2201d68a2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:22.242443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51fdaf-144c-4610-ade4-8f2201d68a2c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:22.245523 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb1c15-d610-4e77-9e7c-0b3a113591d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:22.246169 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb1c15-d610-4e77-9e7c-0b3a113591d7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:22.246454 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb1c15-d610-4e77-9e7c-0b3a113591d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:22.246734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb1c15-d610-4e77-9e7c-0b3a113591d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:22.251247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 239/502] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:22.252604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0213eb-e6f8-4931-badd-a829e70bad37 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:22.252905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0213eb-e6f8-4931-badd-a829e70bad37 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:22.253186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0213eb-e6f8-4931-badd-a829e70bad37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:22.269988 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-df0213eb-e6f8-4931-badd-a829e70bad37 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:22.271212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-df0213eb-e6f8-4931-badd-a829e70bad37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:22.317712 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb1c15-d610-4e77-9e7c-0b3a113591d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:22.319093 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb1c15-d610-4e77-9e7c-0b3a113591d7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:22.339262 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 240/503] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:22.340140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0213eb-e6f8-4931-badd-a829e70bad37 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:22.383457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:22.432405 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:22.438413 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 264/504] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:22.439281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb1c15-d610-4e77-9e7c-0b3a113591d7 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:22.447494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0213eb-e6f8-4931-badd-a829e70bad37 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:22.482814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc73f1-5e69-44b0-bfd6-730746fe414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:22.483604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc73f1-5e69-44b0-bfd6-730746fe414d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:22.483887 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc73f1-5e69-44b0-bfd6-730746fe414d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:22.484196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc73f1-5e69-44b0-bfd6-730746fe414d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:22.494320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc73f1-5e69-44b0-bfd6-730746fe414d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-c4cc73f1-5e69-44b0-bfd6-730746fe414d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:22.502524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:22.503419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:22.504334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:22.504334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:22.504334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:22.504334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:22.504334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:22.504334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:22.504334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:22.504334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:22.504334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:22.504334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:22.504334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:22.504334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:22.504334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 241/505] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:22.505337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc73f1-5e69-44b0-bfd6-730746fe414d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:22.540409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb1c15-d610-4e77-9e7c-0b3a113591d7 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:22.575048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdae27-57c9-47f1-9a33-8564e41ef890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:22.576246 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdae27-57c9-47f1-9a33-8564e41ef890 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:22.576526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdae27-57c9-47f1-9a33-8564e41ef890 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:22.576790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdae27-57c9-47f1-9a33-8564e41ef890 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:22.577618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdae27-57c9-47f1-9a33-8564e41ef890 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:22.577917 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdae27-57c9-47f1-9a33-8564e41ef890 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:22.579371 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdae27-57c9-47f1-9a33-8564e41ef890 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:22.605791 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc73f1-5e69-44b0-bfd6-730746fe414d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:22.615395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 265/506] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jul 21 08:56:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:22.616243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdae27-57c9-47f1-9a33-8564e41ef890 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:22.626319 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:22.644389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa73429-fa87-44d2-9afb-b340177ff60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:22.645183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa73429-fa87-44d2-9afb-b340177ff60b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:22.645474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa73429-fa87-44d2-9afb-b340177ff60b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:22.645756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa73429-fa87-44d2-9afb-b340177ff60b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:22.656568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa73429-fa87-44d2-9afb-b340177ff60b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:22.658055 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa73429-fa87-44d2-9afb-b340177ff60b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:22.666667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:22.677132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-8aa73429-fa87-44d2-9afb-b340177ff60b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:22.677132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:22.677132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:22.677132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:22.677132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:22.677132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:22.677132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:22.677132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:22.677132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:22.677132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:22.677132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:22.677132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:22.677132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:22.677132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:22.677132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:22.678071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:22.678921 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:22.678921 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:22.678921 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:22.678921 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:22.678921 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 242/507] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:22.679228 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa73429-fa87-44d2-9afb-b340177ff60b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:22.726858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdae27-57c9-47f1-9a33-8564e41ef890 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:22.761048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c0dc2-3b24-418f-9486-8c2c46a6a258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:22.761835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c0dc2-3b24-418f-9486-8c2c46a6a258 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:22.762127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c0dc2-3b24-418f-9486-8c2c46a6a258 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:22.762399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c0dc2-3b24-418f-9486-8c2c46a6a258 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:22.773222 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c0dc2-3b24-418f-9486-8c2c46a6a258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:22.774701 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c0dc2-3b24-418f-9486-8c2c46a6a258 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:22.782599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa73429-fa87-44d2-9afb-b340177ff60b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:22.794270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-f76c0dc2-3b24-418f-9486-8c2c46a6a258 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:22.794270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:22.794270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:22.794270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:22.794270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:22.794270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:22.794270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:22.794270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:22.794270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:22.794270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:22.794270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:22.794270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:22.794270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:22.794270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:22.794270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:22.795870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:22.796746 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:22.796746 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:22.796746 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:22.796746 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:22.796746 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 266/508] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:22.796746 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c0dc2-3b24-418f-9486-8c2c46a6a258 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:22.820622 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c846a-c9b3-4f86-b637-b352b4aabc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:22.821436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c846a-c9b3-4f86-b637-b352b4aabc9d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:22.821724 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c846a-c9b3-4f86-b637-b352b4aabc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:22.822002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c846a-c9b3-4f86-b637-b352b4aabc9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:22.835322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c846a-c9b3-4f86-b637-b352b4aabc9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:22.836798 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c846a-c9b3-4f86-b637-b352b4aabc9d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:22.855153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-3f9c846a-c9b3-4f86-b637-b352b4aabc9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:22.855153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:22.855153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:22.855153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:22.855153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:22.855153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:22.855153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:22.855153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:22.855153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:22.855153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:22.855153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:22.855153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:22.855153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:22.855153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:22.855153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:22.856099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:22.856980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:22.856980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:22.856980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:22.856980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:22.856980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 243/509] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:22.857248 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c846a-c9b3-4f86-b637-b352b4aabc9d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:22.863480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:22.906855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c0dc2-3b24-418f-9486-8c2c46a6a258 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:22.913286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:22.940706 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414f04e-3c7a-4cbe-9e62-86e717105c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:22.941371 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414f04e-3c7a-4cbe-9e62-86e717105c1f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:22.941654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414f04e-3c7a-4cbe-9e62-86e717105c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:22.941930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414f04e-3c7a-4cbe-9e62-86e717105c1f None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:22.942694 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414f04e-3c7a-4cbe-9e62-86e717105c1f None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:22.942998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414f04e-3c7a-4cbe-9e62-86e717105c1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:22.944433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414f04e-3c7a-4cbe-9e62-86e717105c1f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:22.952855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 267/510] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:22.953665 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414f04e-3c7a-4cbe-9e62-86e717105c1f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:22.969978 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c846a-c9b3-4f86-b637-b352b4aabc9d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:23.002673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e7646-e84c-41a0-ad5a-187a3426576b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:23.003308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7646-e84c-41a0-ad5a-187a3426576b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:23.003589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7646-e84c-41a0-ad5a-187a3426576b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:23.003862 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7646-e84c-41a0-ad5a-187a3426576b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:23.052180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414f04e-3c7a-4cbe-9e62-86e717105c1f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:23.071287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e7646-e84c-41a0-ad5a-187a3426576b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:23.071558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:23.072846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e7646-e84c-41a0-ad5a-187a3426576b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:23.087224 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4030a4-19fa-4ba4-9411-4cee34b6cd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:23.087822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4030a4-19fa-4ba4-9411-4cee34b6cd03 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:23.088130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4030a4-19fa-4ba4-9411-4cee34b6cd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:23.088408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4030a4-19fa-4ba4-9411-4cee34b6cd03 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:23.089127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4030a4-19fa-4ba4-9411-4cee34b6cd03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:23.089457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4030a4-19fa-4ba4-9411-4cee34b6cd03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:23.090847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4030a4-19fa-4ba4-9411-4cee34b6cd03 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:23.104242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 268/511] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:23.105051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4030a4-19fa-4ba4-9411-4cee34b6cd03 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:23.128513 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:23.183032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 244/512] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:23.184209 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e7646-e84c-41a0-ad5a-187a3426576b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:23.209173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4030a4-19fa-4ba4-9411-4cee34b6cd03 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:23.243707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c33215-5461-4693-97f4-6c4f41a6b375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:23.244395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c33215-5461-4693-97f4-6c4f41a6b375 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:23.244679 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c33215-5461-4693-97f4-6c4f41a6b375 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:23.244957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c33215-5461-4693-97f4-6c4f41a6b375 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:23.245678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c33215-5461-4693-97f4-6c4f41a6b375 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:23.246209 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c33215-5461-4693-97f4-6c4f41a6b375 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:23.249050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c33215-5461-4693-97f4-6c4f41a6b375 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:23.260405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 269/513] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:23.261425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c33215-5461-4693-97f4-6c4f41a6b375 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:23.291055 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e7646-e84c-41a0-ad5a-187a3426576b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:23.303470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:23.328111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4d179-7e07-4adf-aa33-4e4e4bc4af38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:23.328773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4d179-7e07-4adf-aa33-4e4e4bc4af38 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:23.329058 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4d179-7e07-4adf-aa33-4e4e4bc4af38 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:23.329332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4d179-7e07-4adf-aa33-4e4e4bc4af38 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:23.330057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4d179-7e07-4adf-aa33-4e4e4bc4af38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:23.330361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4d179-7e07-4adf-aa33-4e4e4bc4af38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:23.331752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4d179-7e07-4adf-aa33-4e4e4bc4af38 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:23.343781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 245/514] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:23.344577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4d179-7e07-4adf-aa33-4e4e4bc4af38 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:23.360143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:23.367239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c33215-5461-4693-97f4-6c4f41a6b375 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:23.401363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a15665-1784-4bc4-acdb-81a37ab0e2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:23.402154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a15665-1784-4bc4-acdb-81a37ab0e2af None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:23.402472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a15665-1784-4bc4-acdb-81a37ab0e2af None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:23.402804 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a15665-1784-4bc4-acdb-81a37ab0e2af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:23.415332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a15665-1784-4bc4-acdb-81a37ab0e2af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:23.416805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a15665-1784-4bc4-acdb-81a37ab0e2af None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:23.438243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-30a15665-1784-4bc4-acdb-81a37ab0e2af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:23.438243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:23.438243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:23.438243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:23.438243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:23.438243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:23.438243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:23.438243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:23.438243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:23.438243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:23.438243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:23.438243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:23.438243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:23.438243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:23.438243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:23.439180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:23.440059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:23.440059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:23.440059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:23.440059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:23.440059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 270/515] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:23.440333 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a15665-1784-4bc4-acdb-81a37ab0e2af None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:23.453629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4d179-7e07-4adf-aa33-4e4e4bc4af38 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:23.490051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7588043-5052-4bb1-b72e-a6b071aec3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:23.490838 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7588043-5052-4bb1-b72e-a6b071aec3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:23.491120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7588043-5052-4bb1-b72e-a6b071aec3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:23.491394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7588043-5052-4bb1-b72e-a6b071aec3a9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:23.499404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7588043-5052-4bb1-b72e-a6b071aec3a9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:23.500707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7588043-5052-4bb1-b72e-a6b071aec3a9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-e7588043-5052-4bb1-b72e-a6b071aec3a9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:23.517172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:23.518127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:23.519023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:23.519023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:23.519023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:23.519023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:23.519023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:23.519023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:23.519023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:23.519023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:23.519023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:56:23.519023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:23.519023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 246/516] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:23.519582 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7588043-5052-4bb1-b72e-a6b071aec3a9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:23.554392 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:23.564874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a15665-1784-4bc4-acdb-81a37ab0e2af None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:23.593057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:23.598965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6213b-36b9-42c8-94fd-21fd2f03a489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:23.599740 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6213b-36b9-42c8-94fd-21fd2f03a489 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:23.600012 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6213b-36b9-42c8-94fd-21fd2f03a489 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:23.600290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6213b-36b9-42c8-94fd-21fd2f03a489 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:23.612122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6213b-36b9-42c8-94fd-21fd2f03a489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:23.613610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d6213b-36b9-42c8-94fd-21fd2f03a489 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:23.625904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7588043-5052-4bb1-b72e-a6b071aec3a9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:23.631426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-c0d6213b-36b9-42c8-94fd-21fd2f03a489 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:23.631426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:23.631426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:23.631426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:23.631426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:23.631426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:23.631426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:23.631426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:23.631426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:23.631426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:23.631426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:23.631426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:23.631426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:23.631426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:23.631426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:23.632552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:23.633855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:23.633855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:23.633855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:23.633855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:23.633855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 271/517] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:23.633855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6213b-36b9-42c8-94fd-21fd2f03a489 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:23.662210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d4e40-1f42-468a-9515-c83019fa653e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:23.662880 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d4e40-1f42-468a-9515-c83019fa653e None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:23.663165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d4e40-1f42-468a-9515-c83019fa653e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:23.663445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d4e40-1f42-468a-9515-c83019fa653e None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:23.664172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d4e40-1f42-468a-9515-c83019fa653e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:23.665642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d4e40-1f42-468a-9515-c83019fa653e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:23.690662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 247/518] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:56:23 2022] POST /identity/v3/users => generated 247 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:56:23.692218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d4e40-1f42-468a-9515-c83019fa653e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:23.754318 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6213b-36b9-42c8-94fd-21fd2f03a489 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:23.803849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c903ea-97d1-4352-b51f-0f9d5410853f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:23.805093 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c903ea-97d1-4352-b51f-0f9d5410853f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:23.805387 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c903ea-97d1-4352-b51f-0f9d5410853f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:23.805839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c903ea-97d1-4352-b51f-0f9d5410853f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:23.815278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c903ea-97d1-4352-b51f-0f9d5410853f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:23.817061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c903ea-97d1-4352-b51f-0f9d5410853f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:23.824975 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d4e40-1f42-468a-9515-c83019fa653e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:23.833407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-97c903ea-97d1-4352-b51f-0f9d5410853f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:23.833407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:23.833407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:23.833407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:23.833407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:23.833407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:23.833407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:23.833407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:23.833407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:23.833407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:23.833407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:23.833407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:23.833407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:23.833407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:23.833407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:23.834603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:23.835735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:23.835735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:23.835735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:23.835735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:23.835735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:23.835735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:23.835735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:23.835735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:23.835735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 272/519] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:23.837578 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c903ea-97d1-4352-b51f-0f9d5410853f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:23.843124 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:23.867459 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:23.878082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def7351-9582-4075-a929-33af1c147715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:23.878759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def7351-9582-4075-a929-33af1c147715 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:23.879253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def7351-9582-4075-a929-33af1c147715 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:23.879535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def7351-9582-4075-a929-33af1c147715 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:23.880469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def7351-9582-4075-a929-33af1c147715 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:23.880785 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def7351-9582-4075-a929-33af1c147715 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:23.883037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def7351-9582-4075-a929-33af1c147715 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:23.899741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 248/520] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:23.900774 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def7351-9582-4075-a929-33af1c147715 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:23.963157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c903ea-97d1-4352-b51f-0f9d5410853f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:24.003467 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8339b1-a1ae-410e-ba4b-ba6a84151d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:24.004279 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8339b1-a1ae-410e-ba4b-ba6a84151d22 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:24.004591 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8339b1-a1ae-410e-ba4b-ba6a84151d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:24.004832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8339b1-a1ae-410e-ba4b-ba6a84151d22 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:24.006848 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8339b1-a1ae-410e-ba4b-ba6a84151d22 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:24.006848 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8339b1-a1ae-410e-ba4b-ba6a84151d22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:24.007444 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8339b1-a1ae-410e-ba4b-ba6a84151d22 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:24.021294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def7351-9582-4075-a929-33af1c147715 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:24.032227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 273/521] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jul 21 08:56:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:24.033092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8339b1-a1ae-410e-ba4b-ba6a84151d22 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:24.060437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db28d97-d9fe-43d6-b41f-6b2f192ecd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:24.061110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db28d97-d9fe-43d6-b41f-6b2f192ecd46 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:24.061397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db28d97-d9fe-43d6-b41f-6b2f192ecd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:24.061680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db28d97-d9fe-43d6-b41f-6b2f192ecd46 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:24.062415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db28d97-d9fe-43d6-b41f-6b2f192ecd46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:24.062712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db28d97-d9fe-43d6-b41f-6b2f192ecd46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:24.064157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db28d97-d9fe-43d6-b41f-6b2f192ecd46 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:24.079690 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 249/522] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:24.081234 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db28d97-d9fe-43d6-b41f-6b2f192ecd46 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:24.109018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:24.151491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:24.152072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8339b1-a1ae-410e-ba4b-ba6a84151d22 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:24.194762 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d5bdb-4bb6-4d7b-af08-6f4fddd3b5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:24.195886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d5bdb-4bb6-4d7b-af08-6f4fddd3b5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:24.196443 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d5bdb-4bb6-4d7b-af08-6f4fddd3b5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:24.196948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d5bdb-4bb6-4d7b-af08-6f4fddd3b5e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:24.198489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d5bdb-4bb6-4d7b-af08-6f4fddd3b5e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:24.198910 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d5bdb-4bb6-4d7b-af08-6f4fddd3b5e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:24.201778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d5bdb-4bb6-4d7b-af08-6f4fddd3b5e3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:24.212886 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db28d97-d9fe-43d6-b41f-6b2f192ecd46 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:24.236707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 274/523] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:24.238213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d5bdb-4bb6-4d7b-af08-6f4fddd3b5e3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:24.254408 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f898e-9d83-4577-8876-382f4d5a5aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:24.255212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f898e-9d83-4577-8876-382f4d5a5aea None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:24.255492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f898e-9d83-4577-8876-382f4d5a5aea None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:24.256196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f898e-9d83-4577-8876-382f4d5a5aea None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:24.265036 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f898e-9d83-4577-8876-382f4d5a5aea None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:24.266533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f898e-9d83-4577-8876-382f4d5a5aea None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-ac1f898e-9d83-4577-8876-382f4d5a5aea None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:24.281274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:24.282386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:24.283282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:24.283282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:24.283282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:24.283282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:24.283282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:24.283282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:24.283282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:24.283282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:24.283282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 08:56:24.283282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:24.283282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 250/524] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:24.283282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f898e-9d83-4577-8876-382f4d5a5aea None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:24.371440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d5bdb-4bb6-4d7b-af08-6f4fddd3b5e3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:24.405462 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:24.425488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f898e-9d83-4577-8876-382f4d5a5aea None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:24.427296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f22984-530d-4575-9a6d-c757eb653925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:24.429712 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f22984-530d-4575-9a6d-c757eb653925 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:24.430311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f22984-530d-4575-9a6d-c757eb653925 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:24.430897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f22984-530d-4575-9a6d-c757eb653925 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:24.436790 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:24.465685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962beec-5c7a-4fe5-8055-2b4fe3109e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:24.466530 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962beec-5c7a-4fe5-8055-2b4fe3109e2f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:24.466819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962beec-5c7a-4fe5-8055-2b4fe3109e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:24.467128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962beec-5c7a-4fe5-8055-2b4fe3109e2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:24.475781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962beec-5c7a-4fe5-8055-2b4fe3109e2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:24.477324 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962beec-5c7a-4fe5-8055-2b4fe3109e2f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:24.498465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-f962beec-5c7a-4fe5-8055-2b4fe3109e2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:24.498465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:24.498465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:24.498465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:24.498465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:24.498465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:24.498465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:24.498465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:24.498465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:24.498465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:24.498465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:24.498465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:24.498465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:24.498465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:24.498465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:24.499730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:24.500977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:24.500977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:24.500977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:24.500977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:24.500977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:24.500977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:24.500977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:24.500977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:24.500977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 251/525] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:24.500977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962beec-5c7a-4fe5-8055-2b4fe3109e2f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:24.520642 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f22984-530d-4575-9a6d-c757eb653925 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:24.522864 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f22984-530d-4575-9a6d-c757eb653925 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:24.634523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962beec-5c7a-4fe5-8055-2b4fe3109e2f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:24.637128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 275/526] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:24.638095 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f22984-530d-4575-9a6d-c757eb653925 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:24.674614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:24.679931 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c4a74-3299-4c13-9d88-c92a35847b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:24.680736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c4a74-3299-4c13-9d88-c92a35847b7f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:24.681119 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c4a74-3299-4c13-9d88-c92a35847b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:24.681518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c4a74-3299-4c13-9d88-c92a35847b7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:24.682424 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c4a74-3299-4c13-9d88-c92a35847b7f None admin] RBAC: Adding query filter params (role.id=8511cf3e5d2c4acdae6d0ffda5e72c63, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=c48d41dcf9b344a8b2fec77f4dd75fcb) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:24.682801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c4a74-3299-4c13-9d88-c92a35847b7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:24.684355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c4a74-3299-4c13-9d88-c92a35847b7f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:24.696340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 252/527] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:56:24 2022] GET /identity/v3/role_assignments?user.id=c48d41dcf9b344a8b2fec77f4dd75fcb&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:24.697990 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c4a74-3299-4c13-9d88-c92a35847b7f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:24.742368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:24.797032 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f22984-530d-4575-9a6d-c757eb653925 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:24.842375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c4a74-3299-4c13-9d88-c92a35847b7f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:24.843499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9859c3-e7d1-4337-9aa1-75a6dbab9656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:24.844444 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9859c3-e7d1-4337-9aa1-75a6dbab9656 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:24.844900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9859c3-e7d1-4337-9aa1-75a6dbab9656 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:24.845353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9859c3-e7d1-4337-9aa1-75a6dbab9656 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:24.846410 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9859c3-e7d1-4337-9aa1-75a6dbab9656 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:24.846878 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9859c3-e7d1-4337-9aa1-75a6dbab9656 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:24.848587 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9859c3-e7d1-4337-9aa1-75a6dbab9656 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:24.873182 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 276/528] 10.222.0.13 () {56 vars in 1139 bytes} [Thu Jul 21 08:56:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:24.874480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9859c3-e7d1-4337-9aa1-75a6dbab9656 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:24.897488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da919b87-713a-4ed2-9465-d4302b92e06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:24.898146 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da919b87-713a-4ed2-9465-d4302b92e06e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:24.898426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da919b87-713a-4ed2-9465-d4302b92e06e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:24.898698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da919b87-713a-4ed2-9465-d4302b92e06e None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:24.899487 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da919b87-713a-4ed2-9465-d4302b92e06e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:24.899795 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da919b87-713a-4ed2-9465-d4302b92e06e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:24.901297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da919b87-713a-4ed2-9465-d4302b92e06e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:24.922397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 253/529] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:24.923371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da919b87-713a-4ed2-9465-d4302b92e06e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:24.973268 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:24.997109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9859c3-e7d1-4337-9aa1-75a6dbab9656 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:25.029983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:25.039132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7fc99-8a06-46d6-bd86-23eae7dd18a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:25.040233 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7fc99-8a06-46d6-bd86-23eae7dd18a1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:25.040660 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da919b87-713a-4ed2-9465-d4302b92e06e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:25.040865 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7fc99-8a06-46d6-bd86-23eae7dd18a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:25.041281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7fc99-8a06-46d6-bd86-23eae7dd18a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:25.054070 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7fc99-8a06-46d6-bd86-23eae7dd18a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:25.055833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7fc99-8a06-46d6-bd86-23eae7dd18a1 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:25.076944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-efd7fc99-8a06-46d6-bd86-23eae7dd18a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:25.076944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:25.076944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:25.076944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:25.076944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:25.076944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:25.076944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:25.076944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:25.076944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:25.076944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:25.076944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:25.076944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:25.076944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:25.076944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:25.076944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:25.078078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:25.079111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:25.079111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:25.079111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:25.079111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:25.081352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 277/530] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:25.082425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bb6e1-4b37-4998-b433-3280dbd0f6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:25.082819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7fc99-8a06-46d6-bd86-23eae7dd18a1 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:25.083080 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bb6e1-4b37-4998-b433-3280dbd0f6da None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:25.083373 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bb6e1-4b37-4998-b433-3280dbd0f6da None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:25.083650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bb6e1-4b37-4998-b433-3280dbd0f6da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:25.173655 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bb6e1-4b37-4998-b433-3280dbd0f6da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:25.175165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73bb6e1-4b37-4998-b433-3280dbd0f6da None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:25.199455 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7fc99-8a06-46d6-bd86-23eae7dd18a1 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:25.233379 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:25.236955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e729e-56d7-4b07-b728-7a2d1fdb5e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:25.237615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e729e-56d7-4b07-b728-7a2d1fdb5e82 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:25.237898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e729e-56d7-4b07-b728-7a2d1fdb5e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:25.238173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e729e-56d7-4b07-b728-7a2d1fdb5e82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:25.239011 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e729e-56d7-4b07-b728-7a2d1fdb5e82 None admin] RBAC: Adding query filter params (role.id=8511cf3e5d2c4acdae6d0ffda5e72c63, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=e053b6b25a4e4a20a05e6ffa6683222b) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:25.239326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e729e-56d7-4b07-b728-7a2d1fdb5e82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:25.241665 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e729e-56d7-4b07-b728-7a2d1fdb5e82 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:25.248977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 278/531] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:56:25 2022] GET /identity/v3/role_assignments?user.id=e053b6b25a4e4a20a05e6ffa6683222b&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:25.250089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e729e-56d7-4b07-b728-7a2d1fdb5e82 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:25.309744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:25.331277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 254/532] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:25.336902 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bb6e1-4b37-4998-b433-3280dbd0f6da None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:25.371041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e729e-56d7-4b07-b728-7a2d1fdb5e82 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:25.409837 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3b037-7390-41ac-9450-d54fa61fb14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:25.410598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3b037-7390-41ac-9450-d54fa61fb14c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:25.410812 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3b037-7390-41ac-9450-d54fa61fb14c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:25.411432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3b037-7390-41ac-9450-d54fa61fb14c None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:25.412368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3b037-7390-41ac-9450-d54fa61fb14c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:25.412819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3b037-7390-41ac-9450-d54fa61fb14c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:25.414330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3b037-7390-41ac-9450-d54fa61fb14c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:25.428978 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 279/533] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:25.438278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3b037-7390-41ac-9450-d54fa61fb14c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:25.494777 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73bb6e1-4b37-4998-b433-3280dbd0f6da None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:25.520318 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:25.536440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57714e4-cea5-47be-9c38-6ce37f6ffb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:25.537257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57714e4-cea5-47be-9c38-6ce37f6ffb8a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:25.537542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57714e4-cea5-47be-9c38-6ce37f6ffb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:25.537821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57714e4-cea5-47be-9c38-6ce37f6ffb8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:25.551434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57714e4-cea5-47be-9c38-6ce37f6ffb8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:25.552963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57714e4-cea5-47be-9c38-6ce37f6ffb8a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:25.562165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3b037-7390-41ac-9450-d54fa61fb14c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:25.577310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-a57714e4-cea5-47be-9c38-6ce37f6ffb8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:25.577310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:25.577310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:25.577310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:25.577310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:25.577310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:25.577310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:25.577310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:25.577310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:25.577310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:25.577310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:25.577310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:25.577310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:25.577310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:25.577310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:25.578237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:25.579085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:25.579085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:25.579085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:25.579085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:25.579085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 255/534] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:25.593285 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57714e4-cea5-47be-9c38-6ce37f6ffb8a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:25.601245 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df213f3-a4b7-4304-b306-cc932902c178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:25.602152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df213f3-a4b7-4304-b306-cc932902c178 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:25.602440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df213f3-a4b7-4304-b306-cc932902c178 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:25.602744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df213f3-a4b7-4304-b306-cc932902c178 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:25.611461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df213f3-a4b7-4304-b306-cc932902c178 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:25.613100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df213f3-a4b7-4304-b306-cc932902c178 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:25.631023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-5df213f3-a4b7-4304-b306-cc932902c178 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:25.631023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:25.631023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:25.631023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:25.631023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:25.631023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:25.631023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:25.631023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:25.631023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:25.631023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:25.631023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:25.631023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:25.631023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:25.631023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:25.631023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:25.631940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:25.632807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:25.632807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:25.632807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:25.632807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:25.632807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:25.632807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:25.632807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:25.632807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:25.632807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 280/535] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:25.645195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df213f3-a4b7-4304-b306-cc932902c178 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:25.676907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:25.792741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df213f3-a4b7-4304-b306-cc932902c178 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:25.811858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:25.833891 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bca58-8255-4589-9edb-64168053cc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:25.834772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bca58-8255-4589-9edb-64168053cc88 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:25.835059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bca58-8255-4589-9edb-64168053cc88 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:25.835488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bca58-8255-4589-9edb-64168053cc88 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:25.836619 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bca58-8255-4589-9edb-64168053cc88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:25.836933 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bca58-8255-4589-9edb-64168053cc88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:25.838503 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bca58-8255-4589-9edb-64168053cc88 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:25.852153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 281/536] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:25.859443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57714e4-cea5-47be-9c38-6ce37f6ffb8a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:25.861736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bca58-8255-4589-9edb-64168053cc88 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:25.927159 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cd08b-ec9f-40af-a587-653171d7b6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:25.927836 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cd08b-ec9f-40af-a587-653171d7b6df None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:25.928152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cd08b-ec9f-40af-a587-653171d7b6df None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:25.928433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cd08b-ec9f-40af-a587-653171d7b6df None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:25.929164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cd08b-ec9f-40af-a587-653171d7b6df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:25.929474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cd08b-ec9f-40af-a587-653171d7b6df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:25.930882 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cd08b-ec9f-40af-a587-653171d7b6df None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:25.944440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 256/537] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:25.945839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdbe0c-3f79-4495-9aa2-00a3b8fc8516 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:25.946142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdbe0c-3f79-4495-9aa2-00a3b8fc8516 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:25.946520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdbe0c-3f79-4495-9aa2-00a3b8fc8516 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:25.952205 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 257/538] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:25.953215 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdbe0c-3f79-4495-9aa2-00a3b8fc8516 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:26.019111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bca58-8255-4589-9edb-64168053cc88 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:26.074338 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48989ad6-56b4-45bc-a454-54f89b4f7e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:26.075022 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48989ad6-56b4-45bc-a454-54f89b4f7e34 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:26.075398 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48989ad6-56b4-45bc-a454-54f89b4f7e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:26.076346 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48989ad6-56b4-45bc-a454-54f89b4f7e34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:26.077871 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48989ad6-56b4-45bc-a454-54f89b4f7e34 None admin] RBAC: Adding query filter params (role.id=8511cf3e5d2c4acdae6d0ffda5e72c63, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=8a3b3421e9d8427388f4fd2be5a601cd) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:26.078252 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48989ad6-56b4-45bc-a454-54f89b4f7e34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:26.079797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48989ad6-56b4-45bc-a454-54f89b4f7e34 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:26.086352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 282/539] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:56:25 2022] GET /identity/v3/role_assignments?user.id=8a3b3421e9d8427388f4fd2be5a601cd&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:26.087696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf8201-a7ca-4a98-bc0f-a25112ffd154 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:26.088075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf8201-a7ca-4a98-bc0f-a25112ffd154 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:26.088429 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf8201-a7ca-4a98-bc0f-a25112ffd154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:26.100619 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdbe0c-3f79-4495-9aa2-00a3b8fc8516 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:26.103709 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:26.108545 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-9caf8201-a7ca-4a98-bc0f-a25112ffd154 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:26.109927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9caf8201-a7ca-4a98-bc0f-a25112ffd154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:26.145026 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68effc-f68d-4f8f-8d40-69e2366fcfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:26.146256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68effc-f68d-4f8f-8d40-69e2366fcfb9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:26.146719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68effc-f68d-4f8f-8d40-69e2366fcfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:26.147162 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68effc-f68d-4f8f-8d40-69e2366fcfb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:26.151900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:26.157196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68effc-f68d-4f8f-8d40-69e2366fcfb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:26.160168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68effc-f68d-4f8f-8d40-69e2366fcfb9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:26.176156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-3b68effc-f68d-4f8f-8d40-69e2366fcfb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:26.176156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:26.176156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:26.176156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:26.176156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:26.176156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:26.176156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:26.176156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:26.176156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:26.176156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:26.176156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:26.176156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:26.176156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:26.176156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:26.176156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:26.177048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:26.177912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:26.177912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:26.177912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:26.177912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:26.177912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:26.177912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:26.177912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:26.177912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:26.180545 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 258/540] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:26.186539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68effc-f68d-4f8f-8d40-69e2366fcfb9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:26.208430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 283/541] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:26.249580 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf8201-a7ca-4a98-bc0f-a25112ffd154 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:26.349064 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68effc-f68d-4f8f-8d40-69e2366fcfb9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:26.390171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca79923-6bd6-426d-a4c5-1bb098aa6c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:26.391211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca79923-6bd6-426d-a4c5-1bb098aa6c4e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:26.391696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca79923-6bd6-426d-a4c5-1bb098aa6c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:26.392202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca79923-6bd6-426d-a4c5-1bb098aa6c4e None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:26.393361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca79923-6bd6-426d-a4c5-1bb098aa6c4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:26.393871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca79923-6bd6-426d-a4c5-1bb098aa6c4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:26.395620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca79923-6bd6-426d-a4c5-1bb098aa6c4e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:26.411063 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 259/542] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:26.417495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca79923-6bd6-426d-a4c5-1bb098aa6c4e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:26.456727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:26.465732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf8201-a7ca-4a98-bc0f-a25112ffd154 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:26.532795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2ea0d-80bf-41ed-ba9a-c80ce9aa9dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:26.533602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2ea0d-80bf-41ed-ba9a-c80ce9aa9dfd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:26.533885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2ea0d-80bf-41ed-ba9a-c80ce9aa9dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:26.534160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2ea0d-80bf-41ed-ba9a-c80ce9aa9dfd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:26.562070 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2ea0d-80bf-41ed-ba9a-c80ce9aa9dfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-16e2ea0d-80bf-41ed-ba9a-c80ce9aa9dfd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:26.572906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:26.573842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:26.574607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:26.574607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:26.574607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:26.574607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:26.574607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:26.574607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:26.574607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:26.574607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:26.574607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:26.574607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:26.574607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:26.574607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:26.574607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 284/543] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:26.584966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2ea0d-80bf-41ed-ba9a-c80ce9aa9dfd None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:26.586338 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca79923-6bd6-426d-a4c5-1bb098aa6c4e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:26.629481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a996426-2183-4483-92c3-36b0a4868949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:26.630077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a996426-2183-4483-92c3-36b0a4868949 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:26.630360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a996426-2183-4483-92c3-36b0a4868949 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:26.630644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a996426-2183-4483-92c3-36b0a4868949 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:26.631494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a996426-2183-4483-92c3-36b0a4868949 None admin] RBAC: Adding query filter params (role.id=8511cf3e5d2c4acdae6d0ffda5e72c63, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=9e2a13c2cb864175a082de3ca8b166c9) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:26.631871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a996426-2183-4483-92c3-36b0a4868949 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:26.633618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a996426-2183-4483-92c3-36b0a4868949 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:26.641326 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 260/544] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:56:26 2022] GET /identity/v3/role_assignments?user.id=9e2a13c2cb864175a082de3ca8b166c9&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:26.670596 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:26.768844 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2ea0d-80bf-41ed-ba9a-c80ce9aa9dfd None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:26.818722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3079d8-b552-4c4b-9a4c-e2fbc137fef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:26.819794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3079d8-b552-4c4b-9a4c-e2fbc137fef8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:26.820317 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3079d8-b552-4c4b-9a4c-e2fbc137fef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:26.820812 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3079d8-b552-4c4b-9a4c-e2fbc137fef8 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:26.821851 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3079d8-b552-4c4b-9a4c-e2fbc137fef8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:26.822419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3079d8-b552-4c4b-9a4c-e2fbc137fef8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:26.824090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3079d8-b552-4c4b-9a4c-e2fbc137fef8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:26.840621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 285/545] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:26.846020 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a996426-2183-4483-92c3-36b0a4868949 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:26.956252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:26.973949 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a996426-2183-4483-92c3-36b0a4868949 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:27.026079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb7785-c762-47e6-9533-f2da307986fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:27.027245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb7785-c762-47e6-9533-f2da307986fc None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:27.027763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb7785-c762-47e6-9533-f2da307986fc None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:27.028266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb7785-c762-47e6-9533-f2da307986fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:27.043713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb7785-c762-47e6-9533-f2da307986fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:27.045649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb7785-c762-47e6-9533-f2da307986fc None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:27.073452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-a7eb7785-c762-47e6-9533-f2da307986fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:27.073452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:27.073452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:27.073452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:27.073452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:27.073452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:27.073452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:27.073452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:27.073452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:27.073452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:27.073452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:27.073452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:27.073452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:27.073452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:27.073452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:27.074355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:27.075192 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:27.075192 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:27.075192 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:27.075192 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:27.077505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 261/546] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:27.083158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3079d8-b552-4c4b-9a4c-e2fbc137fef8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:27.157814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626475c-015f-4d20-b34a-3caae1f5d15a None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:27.158475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626475c-015f-4d20-b34a-3caae1f5d15a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:27.158957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626475c-015f-4d20-b34a-3caae1f5d15a None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:27.160562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 262/547] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:56:27.165971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d60fa3-17f9-455c-be99-33e35d8a73c0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:27.166259 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d60fa3-17f9-455c-be99-33e35d8a73c0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:27.166535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d60fa3-17f9-455c-be99-33e35d8a73c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:27.204334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-96d60fa3-17f9-455c-be99-33e35d8a73c0 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:27.207475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-96d60fa3-17f9-455c-be99-33e35d8a73c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:27.241049 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:27.264916 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3079d8-b552-4c4b-9a4c-e2fbc137fef8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:27.338780 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 263/548] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:56:27.354790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65b727-7df6-449c-94ae-36e6cae0ffb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:27.356132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65b727-7df6-449c-94ae-36e6cae0ffb6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:27.356796 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65b727-7df6-449c-94ae-36e6cae0ffb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:27.359162 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65b727-7df6-449c-94ae-36e6cae0ffb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:27.360960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65b727-7df6-449c-94ae-36e6cae0ffb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:27.361824 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65b727-7df6-449c-94ae-36e6cae0ffb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:27.364815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65b727-7df6-449c-94ae-36e6cae0ffb6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:27.371760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d60fa3-17f9-455c-be99-33e35d8a73c0 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:27.402601 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 286/549] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:27.408613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65b727-7df6-449c-94ae-36e6cae0ffb6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:27.491937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:27.548672 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65b727-7df6-449c-94ae-36e6cae0ffb6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:27.562744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d60fa3-17f9-455c-be99-33e35d8a73c0 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:27.593592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d44bc9-c76c-41eb-83ca-797369b69045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:27.594402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d44bc9-c76c-41eb-83ca-797369b69045 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:27.594688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d44bc9-c76c-41eb-83ca-797369b69045 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:27.594962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d44bc9-c76c-41eb-83ca-797369b69045 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:27.613122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d44bc9-c76c-41eb-83ca-797369b69045 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:27.614617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d44bc9-c76c-41eb-83ca-797369b69045 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-60d44bc9-c76c-41eb-83ca-797369b69045 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:27.638784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:27.639898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:27.640814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:27.640814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:27.640814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:27.640814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:27.640814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:27.640814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:27.640814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:27.640814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:27.640814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:56:27.640814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:27.640814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 287/550] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:56:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:27.645218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927d883-b852-4b30-92ab-9928223e43ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:27.646631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927d883-b852-4b30-92ab-9928223e43ec None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:27.647120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927d883-b852-4b30-92ab-9928223e43ec None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:27.647592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927d883-b852-4b30-92ab-9928223e43ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:27.648183 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d44bc9-c76c-41eb-83ca-797369b69045 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:27.669339 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927d883-b852-4b30-92ab-9928223e43ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-3927d883-b852-4b30-92ab-9928223e43ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:27.687472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:27.688620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:27.689584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:27.689584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:27.689584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:27.689584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:27.689584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:27.689584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:27.689584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:27.689584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:27.689584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:27.689584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:27.689584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:27.689584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:27.690200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:27.694734 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 264/551] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:27.699955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927d883-b852-4b30-92ab-9928223e43ec None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:27.840978 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927d883-b852-4b30-92ab-9928223e43ec None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:27.876682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:27.882658 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d44bc9-c76c-41eb-83ca-797369b69045 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:27.884488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe70204-57c3-41ad-a3d5-70b1c6cbd9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:27.885264 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe70204-57c3-41ad-a3d5-70b1c6cbd9cf None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:27.885557 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe70204-57c3-41ad-a3d5-70b1c6cbd9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:27.885830 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe70204-57c3-41ad-a3d5-70b1c6cbd9cf None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:27.886584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe70204-57c3-41ad-a3d5-70b1c6cbd9cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:27.886889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe70204-57c3-41ad-a3d5-70b1c6cbd9cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:27.888357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe70204-57c3-41ad-a3d5-70b1c6cbd9cf None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:27.899180 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 265/552] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:27.904969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176acab-4c14-4561-8ca3-4533fdaef9eb None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:27.905533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176acab-4c14-4561-8ca3-4533fdaef9eb None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:27.906016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176acab-4c14-4561-8ca3-4533fdaef9eb None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:27.908030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 266/553] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:27.909171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176acab-4c14-4561-8ca3-4533fdaef9eb None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:27.934060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc626c-0160-4398-8fc6-a80d26e50b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:27.934743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc626c-0160-4398-8fc6-a80d26e50b3b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:27.935028 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc626c-0160-4398-8fc6-a80d26e50b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:27.935849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc626c-0160-4398-8fc6-a80d26e50b3b None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:27.937005 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc626c-0160-4398-8fc6-a80d26e50b3b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:27.937313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc626c-0160-4398-8fc6-a80d26e50b3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:27.938992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc626c-0160-4398-8fc6-a80d26e50b3b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:27.959879 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 288/554] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jul 21 08:56:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:27.961456 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0faf6-f724-45d3-b494-90ab31bfaf7c None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:27.961846 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0faf6-f724-45d3-b494-90ab31bfaf7c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:27.962216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0faf6-f724-45d3-b494-90ab31bfaf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:27.980780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-dcb0faf6-f724-45d3-b494-90ab31bfaf7c None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:27.982603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-dcb0faf6-f724-45d3-b494-90ab31bfaf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:28.076845 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:28.082433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176acab-4c14-4561-8ca3-4533fdaef9eb None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:28.083733 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 289/555] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:28.084620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0faf6-f724-45d3-b494-90ab31bfaf7c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:28.149165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d36c24-1ce8-4cce-a750-fe1969b42d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:28.150245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d36c24-1ce8-4cce-a750-fe1969b42d55 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:28.150537 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d36c24-1ce8-4cce-a750-fe1969b42d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:28.150813 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d36c24-1ce8-4cce-a750-fe1969b42d55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:28.166571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d36c24-1ce8-4cce-a750-fe1969b42d55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:28.168366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d36c24-1ce8-4cce-a750-fe1969b42d55 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:28.193899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-93d36c24-1ce8-4cce-a750-fe1969b42d55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:28.193899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:28.193899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:28.193899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:28.193899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:28.193899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:28.193899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:28.193899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:28.193899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:28.193899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:28.193899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:28.193899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:28.193899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:28.193899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:28.193899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:28.195504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:28.197381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:28.197381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:28.197381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:28.197381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:28.197381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 267/556] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:28.197381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d36c24-1ce8-4cce-a750-fe1969b42d55 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:28.217092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0faf6-f724-45d3-b494-90ab31bfaf7c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:28.267744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ee345-ba45-435a-8d1d-29876d058792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:28.268647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ee345-ba45-435a-8d1d-29876d058792 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:28.269499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ee345-ba45-435a-8d1d-29876d058792 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:28.269790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ee345-ba45-435a-8d1d-29876d058792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:28.272319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:28.346513 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d36c24-1ce8-4cce-a750-fe1969b42d55 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:28.367692 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:28.372221 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ee345-ba45-435a-8d1d-29876d058792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:28.375399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ee345-ba45-435a-8d1d-29876d058792 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:28.390808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f1da9-e4de-4ced-90d7-bdc0562c6acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:28.392119 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f1da9-e4de-4ced-90d7-bdc0562c6acd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:28.392511 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f1da9-e4de-4ced-90d7-bdc0562c6acd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:28.392889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f1da9-e4de-4ced-90d7-bdc0562c6acd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:28.406995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f1da9-e4de-4ced-90d7-bdc0562c6acd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-c78f1da9-e4de-4ced-90d7-bdc0562c6acd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:28.418065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:28.420995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:28.422255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:28.422255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:28.422255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:28.422255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:28.422255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:28.422255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:28.422255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:28.422255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:28.422255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:28.422255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:28.422255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:28.422255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:28.422255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 268/557] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:28.422255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f1da9-e4de-4ced-90d7-bdc0562c6acd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:28.509014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 290/558] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:28.510056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ee345-ba45-435a-8d1d-29876d058792 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:28.551947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f1da9-e4de-4ced-90d7-bdc0562c6acd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:28.555655 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:28.599587 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c039e-70e0-47be-b608-37e505b1cc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:28.600411 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c039e-70e0-47be-b608-37e505b1cc74 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:28.600697 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c039e-70e0-47be-b608-37e505b1cc74 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:28.600975 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c039e-70e0-47be-b608-37e505b1cc74 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:28.601717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c039e-70e0-47be-b608-37e505b1cc74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:28.602032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c039e-70e0-47be-b608-37e505b1cc74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:28.604038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c039e-70e0-47be-b608-37e505b1cc74 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:28.628072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 269/559] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:28.628072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031824c-708d-409e-b26b-0ac88e02e5be None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:28.628072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031824c-708d-409e-b26b-0ac88e02e5be None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:28.628072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031824c-708d-409e-b26b-0ac88e02e5be None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:28.628072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 270/560] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:28.628508 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031824c-708d-409e-b26b-0ac88e02e5be None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:28.644091 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:28.647907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ee345-ba45-435a-8d1d-29876d058792 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:28.686559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bcc55-88d3-4727-b57b-aba21281269f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:28.687393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bcc55-88d3-4727-b57b-aba21281269f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:28.687680 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bcc55-88d3-4727-b57b-aba21281269f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:28.687997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bcc55-88d3-4727-b57b-aba21281269f None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:28.688757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bcc55-88d3-4727-b57b-aba21281269f None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:28.689064 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bcc55-88d3-4727-b57b-aba21281269f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:28.690450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bcc55-88d3-4727-b57b-aba21281269f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:28.699456 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 291/561] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:28.700313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bcc55-88d3-4727-b57b-aba21281269f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:28.750150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031824c-708d-409e-b26b-0ac88e02e5be None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:28.789078 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546935a0-0490-4561-9c99-896788dada8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:28.789980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546935a0-0490-4561-9c99-896788dada8f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:28.790264 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546935a0-0490-4561-9c99-896788dada8f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:28.790593 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546935a0-0490-4561-9c99-896788dada8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:28.802666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546935a0-0490-4561-9c99-896788dada8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:28.804204 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546935a0-0490-4561-9c99-896788dada8f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:28.814871 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bcc55-88d3-4727-b57b-aba21281269f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:28.823359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-546935a0-0490-4561-9c99-896788dada8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:28.823359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:28.823359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:28.823359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:28.823359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:28.823359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:28.823359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:28.823359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:28.823359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:28.823359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:28.823359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:28.823359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:28.823359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:28.823359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:28.823359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:28.824496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:28.825571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:28.825571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:28.825571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:28.825571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:28.828317 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 271/562] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:28.829782 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb4131-296a-4c74-a64a-5713036b35f6 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:28.830072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb4131-296a-4c74-a64a-5713036b35f6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:28.830354 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb4131-296a-4c74-a64a-5713036b35f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:28.849379 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0cfb4131-296a-4c74-a64a-5713036b35f6 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:28.850715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0cfb4131-296a-4c74-a64a-5713036b35f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:28.858341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007bc00-7ad2-4222-9753-cf1afd59240a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:28.859785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007bc00-7ad2-4222-9753-cf1afd59240a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:28.860246 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007bc00-7ad2-4222-9753-cf1afd59240a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:28.860671 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007bc00-7ad2-4222-9753-cf1afd59240a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:28.862710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:28.870769 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007bc00-7ad2-4222-9753-cf1afd59240a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:28.872272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007bc00-7ad2-4222-9753-cf1afd59240a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-4007bc00-7ad2-4222-9753-cf1afd59240a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:28.887741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:28.889483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:28.890417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:28.890417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:28.890417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:28.890417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:28.890417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:28.890417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:28.890417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:28.890417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:28.890417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:56:28.890417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:28.890417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 292/563] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jul 21 08:56:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:28.890417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007bc00-7ad2-4222-9753-cf1afd59240a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:28.918775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:28.931967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 272/564] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:28.931967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb4131-296a-4c74-a64a-5713036b35f6 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:29.007225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007bc00-7ad2-4222-9753-cf1afd59240a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:29.042934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7930d9-535f-47b0-b54b-727599bde2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:29.043797 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb4131-296a-4c74-a64a-5713036b35f6 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:29.044008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7930d9-535f-47b0-b54b-727599bde2bd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:29.044294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7930d9-535f-47b0-b54b-727599bde2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:29.044639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7930d9-535f-47b0-b54b-727599bde2bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:29.056676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7930d9-535f-47b0-b54b-727599bde2bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:29.058314 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7930d9-535f-47b0-b54b-727599bde2bd None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:29.081251 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2240f28-5ae5-443f-a768-a86cb418d710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:29.081625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-7a7930d9-535f-47b0-b54b-727599bde2bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:29.081625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:29.081625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:29.081625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:29.081625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:29.081625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:29.081625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:29.081625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:29.081625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:29.081625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:29.081625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:29.081625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:29.081625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:29.081625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:29.081625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:29.082473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:29.083930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:29.083930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:29.083930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:29.083930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:29.084157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2240f28-5ae5-443f-a768-a86cb418d710 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:29.084157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2240f28-5ae5-443f-a768-a86cb418d710 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:29.084157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2240f28-5ae5-443f-a768-a86cb418d710 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:29.084351 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 293/565] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:29.084351 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7930d9-535f-47b0-b54b-727599bde2bd None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:29.087567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2240f28-5ae5-443f-a768-a86cb418d710 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:29.087887 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2240f28-5ae5-443f-a768-a86cb418d710 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:29.089321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2240f28-5ae5-443f-a768-a86cb418d710 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:29.107023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 273/566] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:29.107023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2240f28-5ae5-443f-a768-a86cb418d710 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:29.116789 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:29.171296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:29.198827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7930d9-535f-47b0-b54b-727599bde2bd None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:29.221811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2240f28-5ae5-443f-a768-a86cb418d710 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:29.234056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1a817-7af3-419c-9b26-d058192b5b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:29.234728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1a817-7af3-419c-9b26-d058192b5b46 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:29.235012 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1a817-7af3-419c-9b26-d058192b5b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:29.235296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1a817-7af3-419c-9b26-d058192b5b46 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:29.236179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1a817-7af3-419c-9b26-d058192b5b46 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:29.236490 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1a817-7af3-419c-9b26-d058192b5b46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:29.237945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1a817-7af3-419c-9b26-d058192b5b46 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:29.255750 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d336c-06b1-4bdf-86f4-5c4daa024d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:29.255965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 294/567] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jul 21 08:56:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:29.256721 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d336c-06b1-4bdf-86f4-5c4daa024d30 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:29.256823 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1a817-7af3-419c-9b26-d058192b5b46 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:29.257133 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d336c-06b1-4bdf-86f4-5c4daa024d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:29.257531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d336c-06b1-4bdf-86f4-5c4daa024d30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:29.269008 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d336c-06b1-4bdf-86f4-5c4daa024d30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-fc4d336c-06b1-4bdf-86f4-5c4daa024d30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:29.276881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:29.277784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:29.278602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:29.278602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:29.278602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:29.278602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:29.278602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:29.278602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:29.278602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:29.278602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:29.278602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:29.278602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:29.278602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:29.278602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:29.278602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 274/568] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:29.279325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d336c-06b1-4bdf-86f4-5c4daa024d30 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:29.348181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:29.359788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1a817-7af3-419c-9b26-d058192b5b46 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:29.384215 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d336c-06b1-4bdf-86f4-5c4daa024d30 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:29.390184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:29.395591 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4ff5c-6dbf-4af3-9b13-30be824fdb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:29.396290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4ff5c-6dbf-4af3-9b13-30be824fdb3e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:29.396574 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4ff5c-6dbf-4af3-9b13-30be824fdb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:29.396850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4ff5c-6dbf-4af3-9b13-30be824fdb3e None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:29.397584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4ff5c-6dbf-4af3-9b13-30be824fdb3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:29.397882 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4ff5c-6dbf-4af3-9b13-30be824fdb3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:29.399251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4ff5c-6dbf-4af3-9b13-30be824fdb3e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:29.410594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 295/569] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:29.411842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf8837-5048-4b50-b9f8-833f410cf747 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:29.412148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf8837-5048-4b50-b9f8-833f410cf747 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:29.412426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf8837-5048-4b50-b9f8-833f410cf747 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:29.413817 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 296/570] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:29.414339 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf8837-5048-4b50-b9f8-833f410cf747 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:29.421810 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e6aeb-79a6-4cb1-acae-3fe3c0442315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:29.422600 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e6aeb-79a6-4cb1-acae-3fe3c0442315 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:29.422881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e6aeb-79a6-4cb1-acae-3fe3c0442315 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:29.423154 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e6aeb-79a6-4cb1-acae-3fe3c0442315 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:29.429970 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e6aeb-79a6-4cb1-acae-3fe3c0442315 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:29.431430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e6aeb-79a6-4cb1-acae-3fe3c0442315 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:29.445983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-3b4e6aeb-79a6-4cb1-acae-3fe3c0442315 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:29.445983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:29.445983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:29.445983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:29.445983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:29.445983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:29.445983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:29.445983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:29.445983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:29.445983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:29.445983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:29.445983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:29.445983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:29.445983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:29.445983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:29.447013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:29.447947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:29.447947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:29.447947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:29.447947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:29.447947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:29.447947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:29.447947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:29.447947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:29.447947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 275/571] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:29.447947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e6aeb-79a6-4cb1-acae-3fe3c0442315 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:29.519255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf8837-5048-4b50-b9f8-833f410cf747 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:29.550743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e6aeb-79a6-4cb1-acae-3fe3c0442315 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:29.553264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309a60a-c0e4-4a30-8120-ecb685778dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:29.554052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309a60a-c0e4-4a30-8120-ecb685778dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:29.554333 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309a60a-c0e4-4a30-8120-ecb685778dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:29.554605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309a60a-c0e4-4a30-8120-ecb685778dd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:29.565353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309a60a-c0e4-4a30-8120-ecb685778dd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:29.566809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309a60a-c0e4-4a30-8120-ecb685778dd4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:29.577628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:29.584409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-b309a60a-c0e4-4a30-8120-ecb685778dd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:29.584409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:29.584409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:29.584409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:29.584409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:29.584409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:29.584409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:29.584409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:29.584409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:29.584409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:29.584409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:29.584409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:29.584409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:29.584409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:29.584409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:29.585421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:29.590997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:29.590997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:29.590997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:29.590997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:29.590997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 297/572] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:29.590997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309a60a-c0e4-4a30-8120-ecb685778dd4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:29.591384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdacfa-40ac-4a95-aab3-2a5f256c2b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:29.591384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdacfa-40ac-4a95-aab3-2a5f256c2b4d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:29.591384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdacfa-40ac-4a95-aab3-2a5f256c2b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:29.591384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdacfa-40ac-4a95-aab3-2a5f256c2b4d None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:29.591768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdacfa-40ac-4a95-aab3-2a5f256c2b4d None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:29.591768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdacfa-40ac-4a95-aab3-2a5f256c2b4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:29.591768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdacfa-40ac-4a95-aab3-2a5f256c2b4d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:29.599906 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 276/573] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:29.601274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95118ae6-6629-4941-9b90-417662f51985 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:29.601566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95118ae6-6629-4941-9b90-417662f51985 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:29.601847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95118ae6-6629-4941-9b90-417662f51985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:29.619877 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-95118ae6-6629-4941-9b90-417662f51985 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:29.621132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-95118ae6-6629-4941-9b90-417662f51985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:29.642780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:29.690500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 277/574] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:29.691352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95118ae6-6629-4941-9b90-417662f51985 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:29.702947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309a60a-c0e4-4a30-8120-ecb685778dd4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:29.750019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2d985-da96-48fd-9abe-d62b0d70cac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:29.750817 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2d985-da96-48fd-9abe-d62b0d70cac4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:29.751105 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2d985-da96-48fd-9abe-d62b0d70cac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:29.751387 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2d985-da96-48fd-9abe-d62b0d70cac4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:29.758991 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2d985-da96-48fd-9abe-d62b0d70cac4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:29.760496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2d985-da96-48fd-9abe-d62b0d70cac4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-1dc2d985-da96-48fd-9abe-d62b0d70cac4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:29.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:29.775760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:29.777071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:29.777071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:29.777071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:29.777071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:29.777071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:29.777071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:29.777071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:29.777071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:29.777071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 08:56:29.777071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:29.777071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 298/575] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:29.777071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2d985-da96-48fd-9abe-d62b0d70cac4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:29.802481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95118ae6-6629-4941-9b90-417662f51985 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:29.820278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:29.838743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd1e09-bc97-420b-94ef-0e4ba327b843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:29.839405 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd1e09-bc97-420b-94ef-0e4ba327b843 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:29.839688 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd1e09-bc97-420b-94ef-0e4ba327b843 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:29.839992 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd1e09-bc97-420b-94ef-0e4ba327b843 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:29.840776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd1e09-bc97-420b-94ef-0e4ba327b843 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:29.841083 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd1e09-bc97-420b-94ef-0e4ba327b843 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:29.843018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd1e09-bc97-420b-94ef-0e4ba327b843 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:29.854505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 278/576] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:29.855303 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd1e09-bc97-420b-94ef-0e4ba327b843 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:29.884636 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2d985-da96-48fd-9abe-d62b0d70cac4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:29.886929 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:29.919840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a558c1-2e0a-4387-baea-c72a8a4580fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:29.920563 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a558c1-2e0a-4387-baea-c72a8a4580fb None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:29.920842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a558c1-2e0a-4387-baea-c72a8a4580fb None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:29.921113 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a558c1-2e0a-4387-baea-c72a8a4580fb None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:29.921840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a558c1-2e0a-4387-baea-c72a8a4580fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:29.922140 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a558c1-2e0a-4387-baea-c72a8a4580fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:29.924146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a558c1-2e0a-4387-baea-c72a8a4580fb None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:29.936977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 299/577] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:29.937793 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a558c1-2e0a-4387-baea-c72a8a4580fb None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:29.959636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd1e09-bc97-420b-94ef-0e4ba327b843 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:29.994087 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355ca21-5962-4bda-9728-214177955d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:29.994875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355ca21-5962-4bda-9728-214177955d1d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:29.995160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355ca21-5962-4bda-9728-214177955d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:29.995454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355ca21-5962-4bda-9728-214177955d1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:30.006877 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355ca21-5962-4bda-9728-214177955d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:30.008398 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355ca21-5962-4bda-9728-214177955d1d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:30.025471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-a355ca21-5962-4bda-9728-214177955d1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:30.025471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:30.025471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:30.025471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:30.025471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:30.025471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:30.025471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:30.025471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:30.025471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:30.025471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:30.025471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:30.025471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:30.025471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:30.025471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:30.025471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:30.026585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:30.027808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:30.027808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:30.027808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:30.027808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:30.027808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 279/578] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:30.027808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355ca21-5962-4bda-9728-214177955d1d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:30.039179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a558c1-2e0a-4387-baea-c72a8a4580fb None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:30.063182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:30.084280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c7bac-048c-46ad-b135-ed43143accd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:30.085078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c7bac-048c-46ad-b135-ed43143accd5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:30.085364 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c7bac-048c-46ad-b135-ed43143accd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:30.085644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c7bac-048c-46ad-b135-ed43143accd5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:30.096592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c7bac-048c-46ad-b135-ed43143accd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-fe3c7bac-048c-46ad-b135-ed43143accd5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:30.104355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:30.105452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:30.111783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:30.111783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:30.111783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:30.111783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:30.111783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:30.111783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:30.111783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:30.111783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:30.111783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:30.111783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:30.111783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:30.111783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:30.111783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 300/579] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:30.111783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c7bac-048c-46ad-b135-ed43143accd5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:30.132092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:30.138408 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355ca21-5962-4bda-9728-214177955d1d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:30.173921 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260d98f-b70c-47d8-b8a6-792a5962b8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:30.174588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260d98f-b70c-47d8-b8a6-792a5962b8cd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:30.174870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260d98f-b70c-47d8-b8a6-792a5962b8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:30.175150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260d98f-b70c-47d8-b8a6-792a5962b8cd None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:30.176024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260d98f-b70c-47d8-b8a6-792a5962b8cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:30.176339 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260d98f-b70c-47d8-b8a6-792a5962b8cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:30.177801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260d98f-b70c-47d8-b8a6-792a5962b8cd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:30.199618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 280/580] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jul 21 08:56:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:30.199618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260d98f-b70c-47d8-b8a6-792a5962b8cd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:30.208996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c7bac-048c-46ad-b135-ed43143accd5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:30.243673 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c0c31-6080-4b43-bb0c-6e99a7e5bb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:30.244356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c0c31-6080-4b43-bb0c-6e99a7e5bb01 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:30.244641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c0c31-6080-4b43-bb0c-6e99a7e5bb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:30.244919 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c0c31-6080-4b43-bb0c-6e99a7e5bb01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:30.245755 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c0c31-6080-4b43-bb0c-6e99a7e5bb01 None admin] RBAC: Adding query filter params (role.id=8511cf3e5d2c4acdae6d0ffda5e72c63, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=061def74ec7f42ce9a174e0944bf499f) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:30.246069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c0c31-6080-4b43-bb0c-6e99a7e5bb01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:30.247533 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c0c31-6080-4b43-bb0c-6e99a7e5bb01 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:30.253227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 301/581] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:56:30 2022] GET /identity/v3/role_assignments?user.id=061def74ec7f42ce9a174e0944bf499f&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:30.254022 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c0c31-6080-4b43-bb0c-6e99a7e5bb01 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:30.298787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:30.313385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260d98f-b70c-47d8-b8a6-792a5962b8cd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:30.351770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe134e-3cad-4660-a51b-c5fba4acaddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:30.352614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe134e-3cad-4660-a51b-c5fba4acaddf None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:30.352916 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe134e-3cad-4660-a51b-c5fba4acaddf None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:30.353169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe134e-3cad-4660-a51b-c5fba4acaddf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:30.360718 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe134e-3cad-4660-a51b-c5fba4acaddf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:30.362207 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe134e-3cad-4660-a51b-c5fba4acaddf None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:30.367693 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:30.377173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-aefe134e-3cad-4660-a51b-c5fba4acaddf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:30.377173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:30.377173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:30.377173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:30.377173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:30.377173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:30.377173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:30.377173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:30.377173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:30.377173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:30.377173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:30.377173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:30.377173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:30.377173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:30.377173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:30.378302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:30.379357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:30.379357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:30.379357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:30.379357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:30.379357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:30.379357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:30.379357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:30.379357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:30.383694 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 281/582] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:30.384342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe134e-3cad-4660-a51b-c5fba4acaddf None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:30.386732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c0c31-6080-4b43-bb0c-6e99a7e5bb01 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:30.429119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77a7af-0766-4fcb-a3ad-ff16a564c81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:30.429789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77a7af-0766-4fcb-a3ad-ff16a564c81e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:30.430076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77a7af-0766-4fcb-a3ad-ff16a564c81e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:30.430883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77a7af-0766-4fcb-a3ad-ff16a564c81e None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:30.431666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77a7af-0766-4fcb-a3ad-ff16a564c81e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:30.431997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77a7af-0766-4fcb-a3ad-ff16a564c81e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:30.434213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77a7af-0766-4fcb-a3ad-ff16a564c81e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:30.456008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 302/583] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:30.456008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77a7af-0766-4fcb-a3ad-ff16a564c81e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:30.495019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe134e-3cad-4660-a51b-c5fba4acaddf None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:30.531498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93be7a5-60c3-4285-988d-bd185528707d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:30.532206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93be7a5-60c3-4285-988d-bd185528707d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:30.532495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93be7a5-60c3-4285-988d-bd185528707d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:30.532778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93be7a5-60c3-4285-988d-bd185528707d None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:30.533536 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93be7a5-60c3-4285-988d-bd185528707d None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:30.533845 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93be7a5-60c3-4285-988d-bd185528707d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:30.535234 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93be7a5-60c3-4285-988d-bd185528707d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:30.549161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 282/584] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:30.549161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93be7a5-60c3-4285-988d-bd185528707d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:30.558893 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:30.591561 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77a7af-0766-4fcb-a3ad-ff16a564c81e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:30.633449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a91ceb-bc60-4d11-a9ef-cc34da066b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:30.634400 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a91ceb-bc60-4d11-a9ef-cc34da066b38 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:30.634783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a91ceb-bc60-4d11-a9ef-cc34da066b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:30.635156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a91ceb-bc60-4d11-a9ef-cc34da066b38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:30.648078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a91ceb-bc60-4d11-a9ef-cc34da066b38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:30.649606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a91ceb-bc60-4d11-a9ef-cc34da066b38 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:30.659295 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:30.670354 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93be7a5-60c3-4285-988d-bd185528707d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:30.670765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-46a91ceb-bc60-4d11-a9ef-cc34da066b38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:30.670765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:30.670765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:30.670765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:30.670765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:30.670765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:30.670765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:30.670765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:30.670765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:30.670765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:30.670765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:30.670765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:30.670765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:30.670765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:30.670765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:30.671627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:30.672524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:30.672524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:30.672524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:30.672524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:30.674813 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 303/585] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:30.675758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a91ceb-bc60-4d11-a9ef-cc34da066b38 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:30.721793 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc334c71-e9ff-4f0d-bbbb-02f722de2c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:30.722341 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc334c71-e9ff-4f0d-bbbb-02f722de2c50 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:30.722627 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc334c71-e9ff-4f0d-bbbb-02f722de2c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:30.722902 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc334c71-e9ff-4f0d-bbbb-02f722de2c50 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:30.724355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc334c71-e9ff-4f0d-bbbb-02f722de2c50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:30.724671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc334c71-e9ff-4f0d-bbbb-02f722de2c50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:30.726504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc334c71-e9ff-4f0d-bbbb-02f722de2c50 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:30.743219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 283/586] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:30.743890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc334c71-e9ff-4f0d-bbbb-02f722de2c50 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:30.794107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a91ceb-bc60-4d11-a9ef-cc34da066b38 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:30.831856 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:30.834418 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59695b4a-dcf5-4547-985a-4099a10f9f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:30.835339 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59695b4a-dcf5-4547-985a-4099a10f9f8a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:30.835619 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59695b4a-dcf5-4547-985a-4099a10f9f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:30.836038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59695b4a-dcf5-4547-985a-4099a10f9f8a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:30.844606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59695b4a-dcf5-4547-985a-4099a10f9f8a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:30.846074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59695b4a-dcf5-4547-985a-4099a10f9f8a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-59695b4a-dcf5-4547-985a-4099a10f9f8a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:30.862842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:30.864510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:30.866779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:30.866779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:30.866779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:30.866779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:30.866779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:30.866779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:30.866779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:30.866779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:30.866779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:56:30.866779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:30.866779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 304/587] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:30.866779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59695b4a-dcf5-4547-985a-4099a10f9f8a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:30.875818 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc334c71-e9ff-4f0d-bbbb-02f722de2c50 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:30.915499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664be2e-7d5b-40ac-892b-41e8c5f90b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:30.916327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664be2e-7d5b-40ac-892b-41e8c5f90b69 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:30.916611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664be2e-7d5b-40ac-892b-41e8c5f90b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:30.916887 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664be2e-7d5b-40ac-892b-41e8c5f90b69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:30.929172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1664be2e-7d5b-40ac-892b-41e8c5f90b69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:30.930667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1664be2e-7d5b-40ac-892b-41e8c5f90b69 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:30.931772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:30.951198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-1664be2e-7d5b-40ac-892b-41e8c5f90b69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:30.951198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:30.951198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:30.951198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:30.951198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:30.951198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:30.951198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:30.951198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:30.951198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:30.951198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:30.951198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:30.951198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:30.951198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:30.951198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:30.951198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:30.952394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:30.954066 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:30.954066 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:30.954066 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:30.954066 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:30.954066 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 284/588] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:30.954958 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664be2e-7d5b-40ac-892b-41e8c5f90b69 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:30.986811 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59695b4a-dcf5-4547-985a-4099a10f9f8a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:31.026636 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a7992-194a-48db-8856-257c77395492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:31.027123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a7992-194a-48db-8856-257c77395492 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:31.027409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a7992-194a-48db-8856-257c77395492 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:31.027678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a7992-194a-48db-8856-257c77395492 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:31.028453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a7992-194a-48db-8856-257c77395492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:31.028758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a7992-194a-48db-8856-257c77395492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:31.030715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a7992-194a-48db-8856-257c77395492 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:31.043975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 305/589] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:31.049305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a7992-194a-48db-8856-257c77395492 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:31.071252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664be2e-7d5b-40ac-892b-41e8c5f90b69 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:31.100286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:31.115703 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce6d23-8c1b-4e8f-8939-eb607e809d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:31.116927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce6d23-8c1b-4e8f-8939-eb607e809d0d None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:31.117175 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce6d23-8c1b-4e8f-8939-eb607e809d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:31.117850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce6d23-8c1b-4e8f-8939-eb607e809d0d None admin] PATH_INFO: `/v3/projects/7fe0f489cbce43859f050c3270e00709/users/c48d41dcf9b344a8b2fec77f4dd75fcb/roles/8511cf3e5d2c4acdae6d0ffda5e72c63` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:31.145449 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce6d23-8c1b-4e8f-8939-eb607e809d0d None admin] RBAC: Authorizing `identity:create_grant(project_id=7fe0f489cbce43859f050c3270e00709, user_id=c48d41dcf9b344a8b2fec77f4dd75fcb, role_id=8511cf3e5d2c4acdae6d0ffda5e72c63)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:31.147439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce6d23-8c1b-4e8f-8939-eb607e809d0d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:31.167647 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 285/590] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:56:30 2022] PUT /identity/v3/projects/7fe0f489cbce43859f050c3270e00709/users/c48d41dcf9b344a8b2fec77f4dd75fcb/roles/8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:56:31.172069 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce6d23-8c1b-4e8f-8939-eb607e809d0d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:31.196806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a7992-194a-48db-8856-257c77395492 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:31.232570 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:31.252890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c452ddb-5534-4189-84a8-24762afc0160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:31.254933 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c452ddb-5534-4189-84a8-24762afc0160 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:31.255523 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c452ddb-5534-4189-84a8-24762afc0160 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:31.256214 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c452ddb-5534-4189-84a8-24762afc0160 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:31.257344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c452ddb-5534-4189-84a8-24762afc0160 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:31.258172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c452ddb-5534-4189-84a8-24762afc0160 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:31.261447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c452ddb-5534-4189-84a8-24762afc0160 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:31.294550 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 306/591] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jul 21 08:56:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:31.296160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c452ddb-5534-4189-84a8-24762afc0160 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:31.310091 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce6d23-8c1b-4e8f-8939-eb607e809d0d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:31.357866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa20147-af8c-48d0-8642-2eec8349eb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:31.358396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa20147-af8c-48d0-8642-2eec8349eb99 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:31.358680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa20147-af8c-48d0-8642-2eec8349eb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:31.358955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa20147-af8c-48d0-8642-2eec8349eb99 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:31.360742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa20147-af8c-48d0-8642-2eec8349eb99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:31.360742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa20147-af8c-48d0-8642-2eec8349eb99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:31.361435 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa20147-af8c-48d0-8642-2eec8349eb99 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:31.376874 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 286/592] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:31.377889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa20147-af8c-48d0-8642-2eec8349eb99 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:31.427154 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:31.484186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c452ddb-5534-4189-84a8-24762afc0160 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:31.529881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf530663-a6fd-4723-9e84-1224bb488786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:31.530742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf530663-a6fd-4723-9e84-1224bb488786 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:31.531492 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf530663-a6fd-4723-9e84-1224bb488786 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:31.531492 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf530663-a6fd-4723-9e84-1224bb488786 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:31.539637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa20147-af8c-48d0-8642-2eec8349eb99 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:31.540286 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf530663-a6fd-4723-9e84-1224bb488786 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:31.541797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf530663-a6fd-4723-9e84-1224bb488786 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:31.558614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-bf530663-a6fd-4723-9e84-1224bb488786 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:31.558614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:31.558614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:31.558614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:31.558614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:31.558614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:31.558614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:31.558614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:31.558614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:31.558614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:31.558614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:31.558614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:31.558614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:31.558614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:31.558614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:31.559635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:31.560688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:31.560688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:31.560688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:31.560688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:31.560688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:31.560688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:31.560688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:31.560688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:31.565449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 307/593] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:31.566474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf530663-a6fd-4723-9e84-1224bb488786 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:31.592859 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d19a09-ab72-4e92-9b48-425049df7daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:31.594948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d19a09-ab72-4e92-9b48-425049df7daa None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:31.595771 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d19a09-ab72-4e92-9b48-425049df7daa None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:31.596339 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d19a09-ab72-4e92-9b48-425049df7daa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:31.615042 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d19a09-ab72-4e92-9b48-425049df7daa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:31.617179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:31.619025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d19a09-ab72-4e92-9b48-425049df7daa None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:31.653389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-a1d19a09-ab72-4e92-9b48-425049df7daa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:31.653389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:31.653389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:31.653389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:31.653389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:31.653389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:31.653389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:31.653389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:31.653389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:31.653389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:31.653389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:31.653389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:31.653389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:31.653389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:31.653389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:31.655050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:31.656663 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:31.656663 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:31.656663 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:31.656663 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:31.664538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 287/594] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:31.664538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d19a09-ab72-4e92-9b48-425049df7daa None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:31.704530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf530663-a6fd-4723-9e84-1224bb488786 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:31.751984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b9eaa-859a-48a1-bb41-4d3c51dd4a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:31.751984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b9eaa-859a-48a1-bb41-4d3c51dd4a6c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:31.751984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b9eaa-859a-48a1-bb41-4d3c51dd4a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:31.751984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b9eaa-859a-48a1-bb41-4d3c51dd4a6c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:31.768789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b9eaa-859a-48a1-bb41-4d3c51dd4a6c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:31.771696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b9eaa-859a-48a1-bb41-4d3c51dd4a6c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-089b9eaa-859a-48a1-bb41-4d3c51dd4a6c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:31.791615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:31.792659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:31.793609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:31.793609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:31.793609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:31.793609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:31.793609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:31.793609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:31.793609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:31.793609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:31.793609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 08:56:31.793609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:31.799008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 308/595] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:31.801577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b9eaa-859a-48a1-bb41-4d3c51dd4a6c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:31.809149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:31.837217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d19a09-ab72-4e92-9b48-425049df7daa None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:31.884833 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1876cc-101d-4717-b83d-c1ca19a1c3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:31.885829 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1876cc-101d-4717-b83d-c1ca19a1c3df None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:31.886263 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1876cc-101d-4717-b83d-c1ca19a1c3df None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:31.886776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1876cc-101d-4717-b83d-c1ca19a1c3df None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:31.888289 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1876cc-101d-4717-b83d-c1ca19a1c3df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:31.889425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1876cc-101d-4717-b83d-c1ca19a1c3df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:31.891953 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1876cc-101d-4717-b83d-c1ca19a1c3df None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:31.912459 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 288/596] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:31.913628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1876cc-101d-4717-b83d-c1ca19a1c3df None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:31.916177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:31.921771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b9eaa-859a-48a1-bb41-4d3c51dd4a6c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:31.959725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28d7f0-76fa-4b01-8f0f-e28469512abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:31.960436 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28d7f0-76fa-4b01-8f0f-e28469512abd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:31.960723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28d7f0-76fa-4b01-8f0f-e28469512abd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:31.961001 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28d7f0-76fa-4b01-8f0f-e28469512abd None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:31.961740 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28d7f0-76fa-4b01-8f0f-e28469512abd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:31.962050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28d7f0-76fa-4b01-8f0f-e28469512abd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:31.963532 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28d7f0-76fa-4b01-8f0f-e28469512abd None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:31.981992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 309/597] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:31.983266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28d7f0-76fa-4b01-8f0f-e28469512abd None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:32.056608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1876cc-101d-4717-b83d-c1ca19a1c3df None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:32.098494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28d7f0-76fa-4b01-8f0f-e28469512abd None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:32.106342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17dce0-c44e-4dec-872d-3586cc70ccd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:32.107457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17dce0-c44e-4dec-872d-3586cc70ccd9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:32.108217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17dce0-c44e-4dec-872d-3586cc70ccd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:32.108868 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b17dce0-c44e-4dec-872d-3586cc70ccd9 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:32.110434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b17dce0-c44e-4dec-872d-3586cc70ccd9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:32.110937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b17dce0-c44e-4dec-872d-3586cc70ccd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:32.113301 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b17dce0-c44e-4dec-872d-3586cc70ccd9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:32.138496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462821f2-1d0d-4016-9975-88dff05dfb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:32.139999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462821f2-1d0d-4016-9975-88dff05dfb81 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:32.140218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462821f2-1d0d-4016-9975-88dff05dfb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:32.140428 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 289/598] 10.222.0.13 () {56 vars in 1139 bytes} [Thu Jul 21 08:56:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:32.140589 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462821f2-1d0d-4016-9975-88dff05dfb81 None admin] PATH_INFO: `/v3/projects/7fe0f489cbce43859f050c3270e00709/users/e053b6b25a4e4a20a05e6ffa6683222b/roles/8511cf3e5d2c4acdae6d0ffda5e72c63` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:32.141973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17dce0-c44e-4dec-872d-3586cc70ccd9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:32.146492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:32.166858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462821f2-1d0d-4016-9975-88dff05dfb81 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fe0f489cbce43859f050c3270e00709, user_id=e053b6b25a4e4a20a05e6ffa6683222b, role_id=8511cf3e5d2c4acdae6d0ffda5e72c63)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:32.168436 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462821f2-1d0d-4016-9975-88dff05dfb81 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:32.188738 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 310/599] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:56:31 2022] PUT /identity/v3/projects/7fe0f489cbce43859f050c3270e00709/users/e053b6b25a4e4a20a05e6ffa6683222b/roles/8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:56:32.190219 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462821f2-1d0d-4016-9975-88dff05dfb81 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:32.207450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:32.312911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b17dce0-c44e-4dec-872d-3586cc70ccd9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:32.343818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462821f2-1d0d-4016-9975-88dff05dfb81 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:32.352560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5d774-cf09-4173-a2ca-6339ff3dee0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:32.353429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5d774-cf09-4173-a2ca-6339ff3dee0b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:32.353719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5d774-cf09-4173-a2ca-6339ff3dee0b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:32.353998 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5d774-cf09-4173-a2ca-6339ff3dee0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:32.362096 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5d774-cf09-4173-a2ca-6339ff3dee0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:32.363580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5d774-cf09-4173-a2ca-6339ff3dee0b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:32.379086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-cbe5d774-cf09-4173-a2ca-6339ff3dee0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:32.379086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:32.379086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:32.379086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:32.379086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:32.379086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:32.379086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:32.379086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:32.379086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:32.379086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:32.379086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:32.379086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:32.379086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:32.379086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:32.379086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:32.380077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:32.380980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:32.380980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:32.380980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:32.380980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:32.380980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:32.380980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:32.380980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:32.380980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:32.380980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 290/600] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:32.386534 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5d774-cf09-4173-a2ca-6339ff3dee0b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:32.399385 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705e9b5-dc7b-4143-a1d4-29756150b175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:32.404132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705e9b5-dc7b-4143-a1d4-29756150b175 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:32.404437 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705e9b5-dc7b-4143-a1d4-29756150b175 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:32.404720 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705e9b5-dc7b-4143-a1d4-29756150b175 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:32.427635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705e9b5-dc7b-4143-a1d4-29756150b175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:32.429474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705e9b5-dc7b-4143-a1d4-29756150b175 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:32.479020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-d705e9b5-dc7b-4143-a1d4-29756150b175 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:32.479020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:32.479020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:32.479020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:32.479020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:32.479020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:32.479020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:32.479020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:32.479020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:32.479020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:32.479020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:32.479020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:32.479020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:32.479020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:32.479020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:32.479990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:32.480870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:32.480870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:32.480870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:32.480870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:32.484440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 311/601] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:32.494941 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705e9b5-dc7b-4143-a1d4-29756150b175 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:32.505975 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:32.565223 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5d774-cf09-4173-a2ca-6339ff3dee0b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:32.582476 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:32.624310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13121393-90e4-4ccc-a3eb-856cbeb05e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:32.626427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13121393-90e4-4ccc-a3eb-856cbeb05e5a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:32.626936 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13121393-90e4-4ccc-a3eb-856cbeb05e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:32.627224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13121393-90e4-4ccc-a3eb-856cbeb05e5a None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:32.628304 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13121393-90e4-4ccc-a3eb-856cbeb05e5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:32.628623 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13121393-90e4-4ccc-a3eb-856cbeb05e5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:32.630799 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13121393-90e4-4ccc-a3eb-856cbeb05e5a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:32.640432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705e9b5-dc7b-4143-a1d4-29756150b175 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:32.651064 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 291/602] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:32.657766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb63c31-065b-4058-bbc8-eb149747c525 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:32.658305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb63c31-065b-4058-bbc8-eb149747c525 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:32.658786 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb63c31-065b-4058-bbc8-eb149747c525 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:32.660772 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 292/603] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:32.661675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb63c31-065b-4058-bbc8-eb149747c525 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:32.684763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5feb275-ba42-4f3d-a7f4-18bbb4137274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:32.686092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5feb275-ba42-4f3d-a7f4-18bbb4137274 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:32.686414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5feb275-ba42-4f3d-a7f4-18bbb4137274 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:32.686784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5feb275-ba42-4f3d-a7f4-18bbb4137274 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:32.687529 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5feb275-ba42-4f3d-a7f4-18bbb4137274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:32.687840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5feb275-ba42-4f3d-a7f4-18bbb4137274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:32.690229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5feb275-ba42-4f3d-a7f4-18bbb4137274 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:32.709021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 312/604] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:32.713243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ee165-5a06-4a2c-b88b-4e7c1d41c767 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:32.713569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ee165-5a06-4a2c-b88b-4e7c1d41c767 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:32.713816 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ee165-5a06-4a2c-b88b-4e7c1d41c767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:32.736839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6c2ee165-5a06-4a2c-b88b-4e7c1d41c767 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:32.738415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6c2ee165-5a06-4a2c-b88b-4e7c1d41c767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:32.790798 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb63c31-065b-4058-bbc8-eb149747c525 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:32.816961 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:32.822139 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 313/605] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:32.824240 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ee165-5a06-4a2c-b88b-4e7c1d41c767 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:32.833014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79a630-6e46-4b54-be51-0beb2d93249c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:32.834026 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79a630-6e46-4b54-be51-0beb2d93249c None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:32.834352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79a630-6e46-4b54-be51-0beb2d93249c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:32.834634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79a630-6e46-4b54-be51-0beb2d93249c None admin] PATH_INFO: `/v3/projects/7fe0f489cbce43859f050c3270e00709/users/8a3b3421e9d8427388f4fd2be5a601cd/roles/8511cf3e5d2c4acdae6d0ffda5e72c63` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:32.852696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79a630-6e46-4b54-be51-0beb2d93249c None admin] RBAC: Authorizing `identity:create_grant(project_id=7fe0f489cbce43859f050c3270e00709, user_id=8a3b3421e9d8427388f4fd2be5a601cd, role_id=8511cf3e5d2c4acdae6d0ffda5e72c63)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:32.854259 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79a630-6e46-4b54-be51-0beb2d93249c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:32.871556 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 293/606] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:56:32 2022] PUT /identity/v3/projects/7fe0f489cbce43859f050c3270e00709/users/8a3b3421e9d8427388f4fd2be5a601cd/roles/8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:56:32.879992 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79a630-6e46-4b54-be51-0beb2d93249c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:32.899669 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:33.006944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ee165-5a06-4a2c-b88b-4e7c1d41c767 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:33.011120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79a630-6e46-4b54-be51-0beb2d93249c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:33.056181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a1c5a-aa64-445b-8e17-974e158de9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:33.059948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a1c5a-aa64-445b-8e17-974e158de9be None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:33.059948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a1c5a-aa64-445b-8e17-974e158de9be None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:33.059948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a1c5a-aa64-445b-8e17-974e158de9be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:33.072675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a1c5a-aa64-445b-8e17-974e158de9be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-f28a1c5a-aa64-445b-8e17-974e158de9be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:33.082399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:33.083390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:33.084257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:33.084257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:33.084257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:33.084257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:33.084257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:33.084257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:33.084257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:33.084257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:33.084257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:33.084257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:33.084257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:33.084257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:33.084257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 294/607] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:33.089670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a1c5a-aa64-445b-8e17-974e158de9be None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:33.102505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c925a-f18a-439f-b89f-37d38adcf902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:33.103653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c925a-f18a-439f-b89f-37d38adcf902 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:33.104123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c925a-f18a-439f-b89f-37d38adcf902 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:33.104563 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c925a-f18a-439f-b89f-37d38adcf902 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:33.121234 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c925a-f18a-439f-b89f-37d38adcf902 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:33.123498 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c925a-f18a-439f-b89f-37d38adcf902 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:33.130409 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:33.152254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-0b6c925a-f18a-439f-b89f-37d38adcf902 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:33.152254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:33.152254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:33.152254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:33.152254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:33.152254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:33.152254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:33.152254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:33.152254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:33.152254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:33.152254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:33.152254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:33.152254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:33.152254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:33.152254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:33.153262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:33.154757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:33.154757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:33.154757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:33.154757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:33.154757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:33.154757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:33.154757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:33.154757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:33.154757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 314/608] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:33.160970 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c925a-f18a-439f-b89f-37d38adcf902 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:33.230535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a1c5a-aa64-445b-8e17-974e158de9be None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:33.283481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd9a20-a3d1-4ab4-bdf9-320910407ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:33.284116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd9a20-a3d1-4ab4-bdf9-320910407ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:33.284402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd9a20-a3d1-4ab4-bdf9-320910407ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:33.284681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd9a20-a3d1-4ab4-bdf9-320910407ca2 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:33.285508 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd9a20-a3d1-4ab4-bdf9-320910407ca2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:33.285817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd9a20-a3d1-4ab4-bdf9-320910407ca2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:33.287460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd9a20-a3d1-4ab4-bdf9-320910407ca2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:33.298767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 295/609] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:33.301388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c925a-f18a-439f-b89f-37d38adcf902 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:33.304917 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd9a20-a3d1-4ab4-bdf9-320910407ca2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:33.314457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:33.343089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55eb12-bb36-4c71-9f90-46140481ecd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:33.343940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55eb12-bb36-4c71-9f90-46140481ecd0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:33.344359 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55eb12-bb36-4c71-9f90-46140481ecd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:33.344737 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55eb12-bb36-4c71-9f90-46140481ecd0 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:33.345644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55eb12-bb36-4c71-9f90-46140481ecd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:33.346056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55eb12-bb36-4c71-9f90-46140481ecd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:33.347815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55eb12-bb36-4c71-9f90-46140481ecd0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:33.362071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 315/610] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:33.369883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55eb12-bb36-4c71-9f90-46140481ecd0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:33.487270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55eb12-bb36-4c71-9f90-46140481ecd0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:33.489649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:33.521915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd9a20-a3d1-4ab4-bdf9-320910407ca2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:33.529461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347754c-369e-457c-9ca8-24c698234a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:33.530544 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347754c-369e-457c-9ca8-24c698234a07 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:33.530898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347754c-369e-457c-9ca8-24c698234a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:33.531253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347754c-369e-457c-9ca8-24c698234a07 None admin] PATH_INFO: `/v3/projects/7fe0f489cbce43859f050c3270e00709/users/9e2a13c2cb864175a082de3ca8b166c9/roles/8511cf3e5d2c4acdae6d0ffda5e72c63` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:33.549885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347754c-369e-457c-9ca8-24c698234a07 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fe0f489cbce43859f050c3270e00709, user_id=9e2a13c2cb864175a082de3ca8b166c9, role_id=8511cf3e5d2c4acdae6d0ffda5e72c63)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:33.551492 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347754c-369e-457c-9ca8-24c698234a07 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:33.567167 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 316/611] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:56:33 2022] PUT /identity/v3/projects/7fe0f489cbce43859f050c3270e00709/users/9e2a13c2cb864175a082de3ca8b166c9/roles/8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:56:33.601904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd758da2-98af-43c4-828f-c96c53981944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:33.602784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd758da2-98af-43c4-828f-c96c53981944 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:33.603147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd758da2-98af-43c4-828f-c96c53981944 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:33.603332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd758da2-98af-43c4-828f-c96c53981944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:33.611763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d3760-c98c-4eb8-b65d-e0ad30da06cd None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:33.612115 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d3760-c98c-4eb8-b65d-e0ad30da06cd None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:33.612356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d3760-c98c-4eb8-b65d-e0ad30da06cd None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:33.613978 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 317/612] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:56:33.620407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b83a0-92c3-4a62-9342-3752bfc0a6de None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:33.620695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b83a0-92c3-4a62-9342-3752bfc0a6de None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:33.620968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b83a0-92c3-4a62-9342-3752bfc0a6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:33.624286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd758da2-98af-43c4-828f-c96c53981944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:33.625740 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd758da2-98af-43c4-828f-c96c53981944 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:33.645009 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-707b83a0-92c3-4a62-9342-3752bfc0a6de None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:33.646274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-707b83a0-92c3-4a62-9342-3752bfc0a6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:33.671577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-cd758da2-98af-43c4-828f-c96c53981944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:33.671577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:33.671577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:33.671577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:33.671577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:33.671577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:33.671577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:33.671577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:33.671577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:33.671577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:33.671577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:33.671577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:33.671577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:33.671577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:33.671577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:33.672566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:33.673433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:33.673433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:33.673433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:33.673433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:33.675090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 296/613] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:33.680530 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd758da2-98af-43c4-828f-c96c53981944 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:33.691780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:33.789809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 318/614] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:56:33.821697 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b83a0-92c3-4a62-9342-3752bfc0a6de None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:33.831890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd758da2-98af-43c4-828f-c96c53981944 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:33.878909 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb7e53-ff74-4d86-9805-67938ba5ff92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:33.879681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb7e53-ff74-4d86-9805-67938ba5ff92 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:33.879851 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb7e53-ff74-4d86-9805-67938ba5ff92 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:33.880158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb7e53-ff74-4d86-9805-67938ba5ff92 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:33.880895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb7e53-ff74-4d86-9805-67938ba5ff92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:33.881202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb7e53-ff74-4d86-9805-67938ba5ff92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:33.882592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb7e53-ff74-4d86-9805-67938ba5ff92 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:33.911524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 297/615] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:33.919271 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb7e53-ff74-4d86-9805-67938ba5ff92 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:33.930798 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:33.997535 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b83a0-92c3-4a62-9342-3752bfc0a6de None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:34.040838 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f897e62-6a8d-43de-9208-9dedd09ffd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:34.041659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f897e62-6a8d-43de-9208-9dedd09ffd2b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:34.041936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f897e62-6a8d-43de-9208-9dedd09ffd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:34.042207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f897e62-6a8d-43de-9208-9dedd09ffd2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:34.066167 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f897e62-6a8d-43de-9208-9dedd09ffd2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-8f897e62-6a8d-43de-9208-9dedd09ffd2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:34.077646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:34.078610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:34.079448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:34.079448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:34.079448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:34.079448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:34.079448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:34.079448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:34.079448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:34.079448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:34.079448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:34.079448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:34.079448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:34.079448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:34.080770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 319/616] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:34.088372 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f897e62-6a8d-43de-9208-9dedd09ffd2b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:34.116138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:34.190542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb7e53-ff74-4d86-9805-67938ba5ff92 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:34.223458 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f897e62-6a8d-43de-9208-9dedd09ffd2b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:34.273751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ddfb3-aad3-47fd-8516-a88fe75dd8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:34.274711 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ddfb3-aad3-47fd-8516-a88fe75dd8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:34.275157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ddfb3-aad3-47fd-8516-a88fe75dd8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:34.275577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ddfb3-aad3-47fd-8516-a88fe75dd8e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:34.276549 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ddfb3-aad3-47fd-8516-a88fe75dd8e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:34.277016 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ddfb3-aad3-47fd-8516-a88fe75dd8e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:34.278799 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ddfb3-aad3-47fd-8516-a88fe75dd8e1 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:34.290372 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 320/617] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:34.297865 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ddfb3-aad3-47fd-8516-a88fe75dd8e1 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:34.314128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cfd70-b06c-4bc2-a70e-ba66ce33b7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:34.315264 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cfd70-b06c-4bc2-a70e-ba66ce33b7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:34.315737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cfd70-b06c-4bc2-a70e-ba66ce33b7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:34.316178 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cfd70-b06c-4bc2-a70e-ba66ce33b7e4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:34.336583 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cfd70-b06c-4bc2-a70e-ba66ce33b7e4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:34.339290 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cfd70-b06c-4bc2-a70e-ba66ce33b7e4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-ae0cfd70-b06c-4bc2-a70e-ba66ce33b7e4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:34.374945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:34.376200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:34.377368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:34.377368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:34.377368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:34.377368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:34.377368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:34.377368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:34.377368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:34.377368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:34.377368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:56:34.377368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:34.385380 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 298/618] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:56:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 467 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:34.392935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cfd70-b06c-4bc2-a70e-ba66ce33b7e4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:34.452864 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:34.514724 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ddfb3-aad3-47fd-8516-a88fe75dd8e1 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:34.545694 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:34.553909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b638f-d668-4062-90a6-67873764d3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:34.556780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b638f-d668-4062-90a6-67873764d3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:34.557068 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b638f-d668-4062-90a6-67873764d3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:34.557343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b638f-d668-4062-90a6-67873764d3c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:34.581667 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b638f-d668-4062-90a6-67873764d3c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:34.583157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b638f-d668-4062-90a6-67873764d3c2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:34.618561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cfd70-b06c-4bc2-a70e-ba66ce33b7e4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:34.622100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-792b638f-d668-4062-90a6-67873764d3c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:34.622100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:34.622100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:34.622100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:34.622100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:34.622100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:34.622100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:34.622100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:34.622100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:34.622100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:34.622100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:34.622100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:34.622100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:34.622100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:34.622100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:34.623827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:34.625069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:34.625069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:34.625069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:34.625069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:34.625722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 321/619] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:34.635089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b638f-d668-4062-90a6-67873764d3c2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:34.667272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca6d29-60b5-48d6-98eb-79e6cf846618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:34.672651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca6d29-60b5-48d6-98eb-79e6cf846618 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:34.673201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca6d29-60b5-48d6-98eb-79e6cf846618 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:34.673677 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca6d29-60b5-48d6-98eb-79e6cf846618 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:34.674736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca6d29-60b5-48d6-98eb-79e6cf846618 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:34.675252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca6d29-60b5-48d6-98eb-79e6cf846618 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:34.676991 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca6d29-60b5-48d6-98eb-79e6cf846618 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:34.722267 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 299/620] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jul 21 08:56:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:34.728028 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca6d29-60b5-48d6-98eb-79e6cf846618 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:34.836996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b638f-d668-4062-90a6-67873764d3c2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:34.871462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca6d29-60b5-48d6-98eb-79e6cf846618 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:34.881571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:34.898515 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:34.902876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ec75c-0836-4ea6-b7fb-34706efb8766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:34.904348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ec75c-0836-4ea6-b7fb-34706efb8766 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:34.905635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ec75c-0836-4ea6-b7fb-34706efb8766 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:34.906283 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ec75c-0836-4ea6-b7fb-34706efb8766 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:34.907618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ec75c-0836-4ea6-b7fb-34706efb8766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:34.908356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ec75c-0836-4ea6-b7fb-34706efb8766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:34.910632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ec75c-0836-4ea6-b7fb-34706efb8766 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:34.912266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e330a99-a996-4bcf-a1eb-a2f3d487b220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:34.913180 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e330a99-a996-4bcf-a1eb-a2f3d487b220 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:34.913547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e330a99-a996-4bcf-a1eb-a2f3d487b220 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:34.913941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e330a99-a996-4bcf-a1eb-a2f3d487b220 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:34.925756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e330a99-a996-4bcf-a1eb-a2f3d487b220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:34.927547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e330a99-a996-4bcf-a1eb-a2f3d487b220 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:34.940385 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 322/621] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:34.942952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbaecd-4255-4d0c-86a8-6393e6d59618 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:34.943643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbaecd-4255-4d0c-86a8-6393e6d59618 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:34.944377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbaecd-4255-4d0c-86a8-6393e6d59618 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:34.947116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-8e330a99-a996-4bcf-a1eb-a2f3d487b220 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:34.947116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:34.947116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:34.947116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:34.947116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:34.947116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:34.947116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:34.947116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:34.947116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:34.947116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:34.947116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:34.947116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:34.947116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:34.947116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:34.947116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:34.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:34.951033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:34.951033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:34.951033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:34.951033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:34.951033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 300/622] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:34.956496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 323/623] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:34 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:34.956761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e330a99-a996-4bcf-a1eb-a2f3d487b220 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:34.957019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fbaecd-4255-4d0c-86a8-6393e6d59618 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:35.093637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e330a99-a996-4bcf-a1eb-a2f3d487b220 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:35.098522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fbaecd-4255-4d0c-86a8-6393e6d59618 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:35.139603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cfade-e18d-4123-bbbb-84269a697d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:35.139603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cfade-e18d-4123-bbbb-84269a697d64 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:35.139603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cfade-e18d-4123-bbbb-84269a697d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:35.139603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cfade-e18d-4123-bbbb-84269a697d64 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:35.151234 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7cfade-e18d-4123-bbbb-84269a697d64 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:35.152825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7cfade-e18d-4123-bbbb-84269a697d64 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:35.155940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f68e5-8f2c-4542-b944-484d4694bed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:35.156861 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f68e5-8f2c-4542-b944-484d4694bed4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:35.157382 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f68e5-8f2c-4542-b944-484d4694bed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:35.157892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f68e5-8f2c-4542-b944-484d4694bed4 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:35.158831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f68e5-8f2c-4542-b944-484d4694bed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:35.159348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f68e5-8f2c-4542-b944-484d4694bed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:35.160976 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f68e5-8f2c-4542-b944-484d4694bed4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-dd7cfade-e18d-4123-bbbb-84269a697d64 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:35.168864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:35.170145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:35.171297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:35.171297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:35.171297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:35.171297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:35.171297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:35.171297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:35.171297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:35.171297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:35.171297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:56:35.171297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:35.176472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 301/624] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jul 21 08:56:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:35.178832 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4da164-b214-48a0-a735-0a1c7bcf235b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:35.179066 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 324/625] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:35.179248 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4da164-b214-48a0-a735-0a1c7bcf235b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:35.179714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4da164-b214-48a0-a735-0a1c7bcf235b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:35.186653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f68e5-8f2c-4542-b944-484d4694bed4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:35.212547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-fa4da164-b214-48a0-a735-0a1c7bcf235b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:35.214464 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fa4da164-b214-48a0-a735-0a1c7bcf235b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:35.244078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:35.258169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:35.306779 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 302/626] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:35.307899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4da164-b214-48a0-a735-0a1c7bcf235b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:35.317598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f68e5-8f2c-4542-b944-484d4694bed4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:35.376494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979327cf-5a8e-4d51-9014-24ae820dde67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:35.377390 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979327cf-5a8e-4d51-9014-24ae820dde67 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:35.377681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979327cf-5a8e-4d51-9014-24ae820dde67 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:35.377956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979327cf-5a8e-4d51-9014-24ae820dde67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:35.386257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979327cf-5a8e-4d51-9014-24ae820dde67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:35.387171 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979327cf-5a8e-4d51-9014-24ae820dde67 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:35.403761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-979327cf-5a8e-4d51-9014-24ae820dde67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:35.403761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:35.403761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:35.403761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:35.403761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:35.403761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:35.403761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:35.403761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:35.403761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:35.403761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:35.403761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:35.403761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:35.403761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:35.403761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:35.403761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:35.405220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:35.406207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:35.406207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:35.406207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:35.406207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:35.406207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:35.406207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:35.406207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:35.406207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:35.406207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 325/627] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:35.406207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979327cf-5a8e-4d51-9014-24ae820dde67 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:35.422266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4da164-b214-48a0-a735-0a1c7bcf235b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:35.461443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78ccf2-3c00-46a8-9a33-32c219d8fdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:35.462449 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78ccf2-3c00-46a8-9a33-32c219d8fdc8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:35.462968 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78ccf2-3c00-46a8-9a33-32c219d8fdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:35.463632 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78ccf2-3c00-46a8-9a33-32c219d8fdc8 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:35.464722 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78ccf2-3c00-46a8-9a33-32c219d8fdc8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:35.465241 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78ccf2-3c00-46a8-9a33-32c219d8fdc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:35.467129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb78ccf2-3c00-46a8-9a33-32c219d8fdc8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:35.492242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 303/628] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jul 21 08:56:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:35.493469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78ccf2-3c00-46a8-9a33-32c219d8fdc8 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:35.524482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:35.529770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979327cf-5a8e-4d51-9014-24ae820dde67 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:35.533690 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:35.569965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419acd2-aa18-4658-806a-92f5d09fbec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:35.570760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419acd2-aa18-4658-806a-92f5d09fbec6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:35.571160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419acd2-aa18-4658-806a-92f5d09fbec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:35.571423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419acd2-aa18-4658-806a-92f5d09fbec6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:35.583661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b419acd2-aa18-4658-806a-92f5d09fbec6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-b419acd2-aa18-4658-806a-92f5d09fbec6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:35.592253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:35.593434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:35.594247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:35.594247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:35.594247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:35.594247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:35.594247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:35.594247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:35.594247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:35.594247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:35.594247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:35.594247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:35.594247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:35.594247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:35.598663 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 326/629] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:35.600006 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419acd2-aa18-4658-806a-92f5d09fbec6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:35.613826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78ccf2-3c00-46a8-9a33-32c219d8fdc8 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:35.651293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524eef55-1d63-4f13-a42b-6f65514909c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:35.652262 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524eef55-1d63-4f13-a42b-6f65514909c4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:35.652687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524eef55-1d63-4f13-a42b-6f65514909c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:35.653103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524eef55-1d63-4f13-a42b-6f65514909c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:35.654186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524eef55-1d63-4f13-a42b-6f65514909c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:35.654736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524eef55-1d63-4f13-a42b-6f65514909c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:35.656557 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524eef55-1d63-4f13-a42b-6f65514909c4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:35.670591 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 304/630] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:35.672139 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524eef55-1d63-4f13-a42b-6f65514909c4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:35.722400 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419acd2-aa18-4658-806a-92f5d09fbec6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:35.758303 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685081d8-16db-43c2-bf89-e4dd392ed576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:35.759007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685081d8-16db-43c2-bf89-e4dd392ed576 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:35.759241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685081d8-16db-43c2-bf89-e4dd392ed576 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:35.759523 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685081d8-16db-43c2-bf89-e4dd392ed576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:35.765715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:35.785232 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524eef55-1d63-4f13-a42b-6f65514909c4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:35.794355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:35.825824 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163ccab-5a22-424f-9386-4510cce0839d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:35.827620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163ccab-5a22-424f-9386-4510cce0839d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:35.827935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163ccab-5a22-424f-9386-4510cce0839d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:35.828234 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163ccab-5a22-424f-9386-4510cce0839d None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:35.828984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163ccab-5a22-424f-9386-4510cce0839d None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:35.829289 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163ccab-5a22-424f-9386-4510cce0839d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:35.830684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163ccab-5a22-424f-9386-4510cce0839d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:35.839305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685081d8-16db-43c2-bf89-e4dd392ed576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:35.839591 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 305/631] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:35.840924 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fa4c5-103c-46e0-a2d7-51373dbcd573 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:35.841215 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fa4c5-103c-46e0-a2d7-51373dbcd573 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:35.841496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fa4c5-103c-46e0-a2d7-51373dbcd573 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:35.841824 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685081d8-16db-43c2-bf89-e4dd392ed576 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:35.847849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 306/632] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:35.847849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fa4c5-103c-46e0-a2d7-51373dbcd573 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:35.961016 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 327/633] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:35.962844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fa4c5-103c-46e0-a2d7-51373dbcd573 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:35.966103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685081d8-16db-43c2-bf89-e4dd392ed576 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:36.001693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb68707-5f1b-4817-9a02-d157275110ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:36.002585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb68707-5f1b-4817-9a02-d157275110ea None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:36.002869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb68707-5f1b-4817-9a02-d157275110ea None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:36.003146 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb68707-5f1b-4817-9a02-d157275110ea None admin] PATH_INFO: `/v3/projects/7fe0f489cbce43859f050c3270e00709/users/061def74ec7f42ce9a174e0944bf499f/roles/8511cf3e5d2c4acdae6d0ffda5e72c63` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:36.018982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb68707-5f1b-4817-9a02-d157275110ea None admin] RBAC: Authorizing `identity:create_grant(project_id=7fe0f489cbce43859f050c3270e00709, user_id=061def74ec7f42ce9a174e0944bf499f, role_id=8511cf3e5d2c4acdae6d0ffda5e72c63)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:36.020496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb68707-5f1b-4817-9a02-d157275110ea None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:36.023528 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:36.034262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:36.036534 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 307/634] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:56:35 2022] PUT /identity/v3/projects/7fe0f489cbce43859f050c3270e00709/users/061def74ec7f42ce9a174e0944bf499f/roles/8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:56:36.039051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d588f-31f8-41c3-bdb7-1ae712b74fe8 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:36.039590 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d588f-31f8-41c3-bdb7-1ae712b74fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:36.040357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d588f-31f8-41c3-bdb7-1ae712b74fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:36.066848 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4f9d588f-31f8-41c3-bdb7-1ae712b74fe8 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:36.068183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4f9d588f-31f8-41c3-bdb7-1ae712b74fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:36.102433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685081d8-16db-43c2-bf89-e4dd392ed576 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:36.150417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9c91d-82a3-4f7e-8fb1-e391099d30bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:36.151227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9c91d-82a3-4f7e-8fb1-e391099d30bd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:36.151515 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9c91d-82a3-4f7e-8fb1-e391099d30bd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:36.151790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9c91d-82a3-4f7e-8fb1-e391099d30bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:36.164277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 308/635] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:36.164277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d588f-31f8-41c3-bdb7-1ae712b74fe8 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:36.175292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9c91d-82a3-4f7e-8fb1-e391099d30bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:36.177027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9c91d-82a3-4f7e-8fb1-e391099d30bd None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:36.198415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-26b9c91d-82a3-4f7e-8fb1-e391099d30bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:36.198415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:36.198415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:36.198415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:36.198415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:36.198415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:36.198415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:36.198415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:36.198415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:36.198415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:36.198415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:36.198415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:36.198415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:36.198415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:36.198415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:36.199530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:36.200617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:36.200617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:36.200617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:36.200617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:36.202322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 328/636] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:36.206723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da36ef2-f544-4387-8175-6187ad9888a2 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:36.206723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da36ef2-f544-4387-8175-6187ad9888a2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:36.206723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da36ef2-f544-4387-8175-6187ad9888a2 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:36.206723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 329/637] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:36.206723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da36ef2-f544-4387-8175-6187ad9888a2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:36.292499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d588f-31f8-41c3-bdb7-1ae712b74fe8 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:36.321442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:36.328935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da36ef2-f544-4387-8175-6187ad9888a2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:36.332074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576fe03-cded-4258-9179-9c52080207d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:36.333013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576fe03-cded-4258-9179-9c52080207d4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:36.333396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576fe03-cded-4258-9179-9c52080207d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:36.333775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576fe03-cded-4258-9179-9c52080207d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:36.346006 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576fe03-cded-4258-9179-9c52080207d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:36.347500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576fe03-cded-4258-9179-9c52080207d4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:36.349327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:36.366761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-7576fe03-cded-4258-9179-9c52080207d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:36.366761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:36.366761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:36.366761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:36.366761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:36.366761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:36.366761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:36.366761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:36.366761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:36.366761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:36.366761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:36.366761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:36.366761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:36.366761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:36.366761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:36.367735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:36.368672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:36.368672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:36.368672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:36.368672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:36.370274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba0a2c-992e-4ce0-be84-96f7dd11940b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:36.371069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba0a2c-992e-4ce0-be84-96f7dd11940b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:36.371350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba0a2c-992e-4ce0-be84-96f7dd11940b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:36.371622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba0a2c-992e-4ce0-be84-96f7dd11940b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:36.372620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 309/638] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:36.374290 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576fe03-cded-4258-9179-9c52080207d4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:36.384904 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba0a2c-992e-4ce0-be84-96f7dd11940b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-beba0a2c-992e-4ce0-be84-96f7dd11940b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:36.394138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:36.395159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:36.396065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:36.396065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:36.396065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:36.396065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:36.396065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:36.396065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:36.396065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:36.396065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:36.396065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:36.396065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:36.396065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:36.396065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:36.396065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 330/639] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:36.397725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93de82-0dbb-4b6d-b881-509bad3c0a93 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:36.397725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93de82-0dbb-4b6d-b881-509bad3c0a93 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:36.397725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93de82-0dbb-4b6d-b881-509bad3c0a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:36.415242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-bb93de82-0dbb-4b6d-b881-509bad3c0a93 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:36.416459 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-bb93de82-0dbb-4b6d-b881-509bad3c0a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:36.490765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576fe03-cded-4258-9179-9c52080207d4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:36.500037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 331/640] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:36.500037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93de82-0dbb-4b6d-b881-509bad3c0a93 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:36.541156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f0874-aa58-422c-bb9b-4d1a1b38ee36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:36.541823 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f0874-aa58-422c-bb9b-4d1a1b38ee36 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:36.542107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f0874-aa58-422c-bb9b-4d1a1b38ee36 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:36.542379 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f0874-aa58-422c-bb9b-4d1a1b38ee36 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:36.543155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f0874-aa58-422c-bb9b-4d1a1b38ee36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:36.543409 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f0874-aa58-422c-bb9b-4d1a1b38ee36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:36.545567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f0874-aa58-422c-bb9b-4d1a1b38ee36 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:36.558790 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 310/641] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:36.559571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f0874-aa58-422c-bb9b-4d1a1b38ee36 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:36.609374 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:36.616825 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93de82-0dbb-4b6d-b881-509bad3c0a93 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:36.625951 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:36.663263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06461cd8-d9f6-4ff1-9ead-9de0c6e84140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:36.664020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06461cd8-d9f6-4ff1-9ead-9de0c6e84140 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:36.664293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06461cd8-d9f6-4ff1-9ead-9de0c6e84140 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:36.664504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06461cd8-d9f6-4ff1-9ead-9de0c6e84140 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:36.665235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06461cd8-d9f6-4ff1-9ead-9de0c6e84140 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:36.665541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06461cd8-d9f6-4ff1-9ead-9de0c6e84140 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:36.666918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06461cd8-d9f6-4ff1-9ead-9de0c6e84140 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:36.683209 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 332/642] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:36.683209 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06461cd8-d9f6-4ff1-9ead-9de0c6e84140 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:36.689859 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f0874-aa58-422c-bb9b-4d1a1b38ee36 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:36.733419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819b9f4-fd7a-44a7-a088-e57e8d593a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:36.734393 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819b9f4-fd7a-44a7-a088-e57e8d593a0b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:36.734670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819b9f4-fd7a-44a7-a088-e57e8d593a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:36.734943 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819b9f4-fd7a-44a7-a088-e57e8d593a0b None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:36.736263 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819b9f4-fd7a-44a7-a088-e57e8d593a0b None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:36.736575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819b9f4-fd7a-44a7-a088-e57e8d593a0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:36.738554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819b9f4-fd7a-44a7-a088-e57e8d593a0b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:36.754426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 311/643] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:36.754426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819b9f4-fd7a-44a7-a088-e57e8d593a0b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:36.804132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06461cd8-d9f6-4ff1-9ead-9de0c6e84140 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:36.845981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fe5fc-2a7b-4084-844a-71a2354f4b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:36.846868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fe5fc-2a7b-4084-844a-71a2354f4b4f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:36.847323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fe5fc-2a7b-4084-844a-71a2354f4b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:36.847600 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fe5fc-2a7b-4084-844a-71a2354f4b4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:36.860657 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92fe5fc-2a7b-4084-844a-71a2354f4b4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-f92fe5fc-2a7b-4084-844a-71a2354f4b4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:36.868381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:36.869451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:36.870393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:36.870393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:36.870393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:36.870393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:36.870393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:36.870393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:36.870393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:36.870393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:36.870393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:36.870393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:36.870393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:36.870393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:36.870393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 333/644] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:36.871176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fe5fc-2a7b-4084-844a-71a2354f4b4f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:36.879965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:36.881368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819b9f4-fd7a-44a7-a088-e57e8d593a0b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:36.914947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:36.923837 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39974d-7aa0-4a5e-9e9b-74887a06f876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:36.924941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39974d-7aa0-4a5e-9e9b-74887a06f876 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:36.925227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39974d-7aa0-4a5e-9e9b-74887a06f876 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:36.925506 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39974d-7aa0-4a5e-9e9b-74887a06f876 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:36.934580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39974d-7aa0-4a5e-9e9b-74887a06f876 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:36.936882 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39974d-7aa0-4a5e-9e9b-74887a06f876 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-0f39974d-7aa0-4a5e-9e9b-74887a06f876 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:36.952903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:36.954106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:36.955437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:36.955437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:36.955437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:36.955437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:36.955437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:36.955437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:36.955437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:36.955437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:36.955437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 08:56:36.955437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:36.955437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 312/645] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:36.955437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39974d-7aa0-4a5e-9e9b-74887a06f876 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:37.011585 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fe5fc-2a7b-4084-844a-71a2354f4b4f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:37.049138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ccbbb-f09c-470e-91ff-007a141fea91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:37.050072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ccbbb-f09c-470e-91ff-007a141fea91 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:37.050369 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ccbbb-f09c-470e-91ff-007a141fea91 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:37.050651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ccbbb-f09c-470e-91ff-007a141fea91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:37.058261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ccbbb-f09c-470e-91ff-007a141fea91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:37.059657 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ccbbb-f09c-470e-91ff-007a141fea91 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:37.075411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-3d1ccbbb-f09c-470e-91ff-007a141fea91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:37.075411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:37.075411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:37.075411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:37.075411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:37.075411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:37.075411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:37.075411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:37.075411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:37.075411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:37.075411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:37.075411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:37.075411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:37.075411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:37.075411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:37.077832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:37.079038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:37.079038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:37.079038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:37.079038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:37.079038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:37.079038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:37.079038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:37.079038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:37.080264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 334/646] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:37.081552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ccbbb-f09c-470e-91ff-007a141fea91 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:37.090342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39974d-7aa0-4a5e-9e9b-74887a06f876 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:37.129257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12839488-8ea3-4a49-9fe6-030bf07dce3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:37.130051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12839488-8ea3-4a49-9fe6-030bf07dce3b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:37.130331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12839488-8ea3-4a49-9fe6-030bf07dce3b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:37.130607 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12839488-8ea3-4a49-9fe6-030bf07dce3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:37.143627 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12839488-8ea3-4a49-9fe6-030bf07dce3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:37.145147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12839488-8ea3-4a49-9fe6-030bf07dce3b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:37.162969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-12839488-8ea3-4a49-9fe6-030bf07dce3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:37.162969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:37.162969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:37.162969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:37.162969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:37.162969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:37.162969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:37.162969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:37.162969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:37.162969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:37.162969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:37.162969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:37.162969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:37.162969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:37.162969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:37.164174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:37.165053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:37.165053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:37.165053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:37.165053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:37.165053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 313/647] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:37.165053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12839488-8ea3-4a49-9fe6-030bf07dce3b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:37.188504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:37.192363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:37.211366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ccbbb-f09c-470e-91ff-007a141fea91 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:37.251088 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e8e23-271e-4be3-b7f4-e6344a382f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:37.251760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e8e23-271e-4be3-b7f4-e6344a382f48 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:37.252139 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e8e23-271e-4be3-b7f4-e6344a382f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:37.252413 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e8e23-271e-4be3-b7f4-e6344a382f48 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:37.253164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e8e23-271e-4be3-b7f4-e6344a382f48 None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:37.253467 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e8e23-271e-4be3-b7f4-e6344a382f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:37.254852 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e8e23-271e-4be3-b7f4-e6344a382f48 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:37.263317 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 335/648] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:37.264699 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e8e23-271e-4be3-b7f4-e6344a382f48 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:37.292306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12839488-8ea3-4a49-9fe6-030bf07dce3b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:37.330610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785371c6-e9eb-4aff-910b-d74afb6cd76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:37.331189 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785371c6-e9eb-4aff-910b-d74afb6cd76f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:37.331469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785371c6-e9eb-4aff-910b-d74afb6cd76f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:37.331739 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785371c6-e9eb-4aff-910b-d74afb6cd76f None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:37.332605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785371c6-e9eb-4aff-910b-d74afb6cd76f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:37.332923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785371c6-e9eb-4aff-910b-d74afb6cd76f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:37.334366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785371c6-e9eb-4aff-910b-d74afb6cd76f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:37.353101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 314/649] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jul 21 08:56:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:37.354019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785371c6-e9eb-4aff-910b-d74afb6cd76f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:37.383631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e8e23-271e-4be3-b7f4-e6344a382f48 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:37.422882 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc8d1f-1c00-4618-9f93-0f25911bcc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:37.424124 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc8d1f-1c00-4618-9f93-0f25911bcc2c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:37.424608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc8d1f-1c00-4618-9f93-0f25911bcc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:37.425086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc8d1f-1c00-4618-9f93-0f25911bcc2c None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:37.426120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc8d1f-1c00-4618-9f93-0f25911bcc2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:37.426596 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc8d1f-1c00-4618-9f93-0f25911bcc2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:37.428307 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc8d1f-1c00-4618-9f93-0f25911bcc2c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:37.432609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:37.447616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 336/650] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:37.448633 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc8d1f-1c00-4618-9f93-0f25911bcc2c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:37.459702 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:37.472681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785371c6-e9eb-4aff-910b-d74afb6cd76f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:37.516689 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b41e92-6171-4593-b7bf-2ba041a0fb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:37.517666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b41e92-6171-4593-b7bf-2ba041a0fb50 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:37.517960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b41e92-6171-4593-b7bf-2ba041a0fb50 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:37.518239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b41e92-6171-4593-b7bf-2ba041a0fb50 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:37.519020 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b41e92-6171-4593-b7bf-2ba041a0fb50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:37.519335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b41e92-6171-4593-b7bf-2ba041a0fb50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:37.520794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b41e92-6171-4593-b7bf-2ba041a0fb50 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:37.533424 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 315/651] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:37.534877 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b41e92-6171-4593-b7bf-2ba041a0fb50 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:37.556864 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc8d1f-1c00-4618-9f93-0f25911bcc2c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:37.594964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78673c0d-9661-43c6-b770-f1a10747f269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:37.595675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78673c0d-9661-43c6-b770-f1a10747f269 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:37.595980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78673c0d-9661-43c6-b770-f1a10747f269 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:37.596256 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78673c0d-9661-43c6-b770-f1a10747f269 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:37.607233 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78673c0d-9661-43c6-b770-f1a10747f269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:37.609714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78673c0d-9661-43c6-b770-f1a10747f269 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:37.631176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-78673c0d-9661-43c6-b770-f1a10747f269 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:37.631176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:37.631176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:37.631176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:37.631176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:37.631176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:37.631176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:37.631176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:37.631176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:37.631176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:37.631176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:37.631176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:37.631176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:37.631176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:37.631176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:37.632250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:37.633108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:37.633108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:37.633108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:37.633108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:37.633108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 337/652] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:37.633108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78673c0d-9661-43c6-b770-f1a10747f269 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:37.652437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b41e92-6171-4593-b7bf-2ba041a0fb50 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:37.684256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:37.691022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c16d54-039c-42d9-a9e0-96cb9763aa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:37.691680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c16d54-039c-42d9-a9e0-96cb9763aa2c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:37.691989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c16d54-039c-42d9-a9e0-96cb9763aa2c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:37.692270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c16d54-039c-42d9-a9e0-96cb9763aa2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:37.702533 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:37.750476 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78673c0d-9661-43c6-b770-f1a10747f269 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:37.771252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c16d54-039c-42d9-a9e0-96cb9763aa2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:37.772763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c16d54-039c-42d9-a9e0-96cb9763aa2c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:37.795812 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f7368-65e2-4386-97e1-980810e5da3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:37.796657 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f7368-65e2-4386-97e1-980810e5da3d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:37.797319 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f7368-65e2-4386-97e1-980810e5da3d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:37.797319 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f7368-65e2-4386-97e1-980810e5da3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:37.798033 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f7368-65e2-4386-97e1-980810e5da3d None admin] RBAC: Adding query filter params (role.id=8511cf3e5d2c4acdae6d0ffda5e72c63, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=c48d41dcf9b344a8b2fec77f4dd75fcb) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:37.798341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f7368-65e2-4386-97e1-980810e5da3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:37.799776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f7368-65e2-4386-97e1-980810e5da3d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:37.808068 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 338/653] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:56:37 2022] GET /identity/v3/role_assignments?user.id=c48d41dcf9b344a8b2fec77f4dd75fcb&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:37.808915 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f7368-65e2-4386-97e1-980810e5da3d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:37.910566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 316/654] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:37.911890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c16d54-039c-42d9-a9e0-96cb9763aa2c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:37.946897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:37.962362 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f7368-65e2-4386-97e1-980810e5da3d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:37.995652 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:38.010682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0acb7-2f1f-4de7-8a5d-26ecbf6cb457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:38.011483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0acb7-2f1f-4de7-8a5d-26ecbf6cb457 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:38.011773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0acb7-2f1f-4de7-8a5d-26ecbf6cb457 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:38.012158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0acb7-2f1f-4de7-8a5d-26ecbf6cb457 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:38.022063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0acb7-2f1f-4de7-8a5d-26ecbf6cb457 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:38.023557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0acb7-2f1f-4de7-8a5d-26ecbf6cb457 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-11b0acb7-2f1f-4de7-8a5d-26ecbf6cb457 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:38.038638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:38.039632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:38.040990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:38.040990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:38.040990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:38.040990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:38.040990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:38.040990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:38.040990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:38.040990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:38.040990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:56:38.040990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:38.040990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 339/655] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:38.041582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0acb7-2f1f-4de7-8a5d-26ecbf6cb457 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:38.053483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c16d54-039c-42d9-a9e0-96cb9763aa2c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:38.115111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b5141-e882-481b-adab-153cc2e1e692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:38.115774 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b5141-e882-481b-adab-153cc2e1e692 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:38.116075 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b5141-e882-481b-adab-153cc2e1e692 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:38.116721 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b5141-e882-481b-adab-153cc2e1e692 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:38.117099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b5141-e882-481b-adab-153cc2e1e692 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:38.117409 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b5141-e882-481b-adab-153cc2e1e692 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:38.118819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b5141-e882-481b-adab-153cc2e1e692 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:38.134466 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 317/656] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:38.134466 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b5141-e882-481b-adab-153cc2e1e692 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:38.179718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0acb7-2f1f-4de7-8a5d-26ecbf6cb457 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:38.216063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0c6b2-4293-4756-9812-3c8e144464d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:38.218065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0c6b2-4293-4756-9812-3c8e144464d1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:38.218673 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0c6b2-4293-4756-9812-3c8e144464d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:38.219123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0c6b2-4293-4756-9812-3c8e144464d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:38.235106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0c6b2-4293-4756-9812-3c8e144464d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:38.236785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0c6b2-4293-4756-9812-3c8e144464d1 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:38.237466 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:38.248741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b5141-e882-481b-adab-153cc2e1e692 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:38.255079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-bfc0c6b2-4293-4756-9812-3c8e144464d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:38.255079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:38.255079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:38.255079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:38.255079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:38.255079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:38.255079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:38.255079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:38.255079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:38.255079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:38.255079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:38.255079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:38.255079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:38.255079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:38.255079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:38.256150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:38.257292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:38.257292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:38.257292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:38.257292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:38.262850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 340/657] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:38.262850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0c6b2-4293-4756-9812-3c8e144464d1 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:38.285543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8932c49-888f-4995-8baf-9aef131188ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:38.286313 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8932c49-888f-4995-8baf-9aef131188ff None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:38.286679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8932c49-888f-4995-8baf-9aef131188ff None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:38.287021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8932c49-888f-4995-8baf-9aef131188ff None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:38.287867 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8932c49-888f-4995-8baf-9aef131188ff None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:38.288213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8932c49-888f-4995-8baf-9aef131188ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:38.289663 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8932c49-888f-4995-8baf-9aef131188ff None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:38.303205 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:38.308573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 318/658] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jul 21 08:56:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:38.309860 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805ed20-7e2f-4c03-9ba6-b74221298d87 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:38.310159 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805ed20-7e2f-4c03-9ba6-b74221298d87 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:38.310438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805ed20-7e2f-4c03-9ba6-b74221298d87 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:38.316746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 319/659] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:38.317606 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805ed20-7e2f-4c03-9ba6-b74221298d87 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:38.378039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0c6b2-4293-4756-9812-3c8e144464d1 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:38.411849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155875b5-6b4e-4d08-a7d8-9411cbc6d9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:38.412776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155875b5-6b4e-4d08-a7d8-9411cbc6d9be None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:38.413125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155875b5-6b4e-4d08-a7d8-9411cbc6d9be None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:38.413467 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155875b5-6b4e-4d08-a7d8-9411cbc6d9be None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:38.421881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155875b5-6b4e-4d08-a7d8-9411cbc6d9be None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:38.423445 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155875b5-6b4e-4d08-a7d8-9411cbc6d9be None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:38.430107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805ed20-7e2f-4c03-9ba6-b74221298d87 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-155875b5-6b4e-4d08-a7d8-9411cbc6d9be None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:38.438674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:38.439682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:38.440668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:38.440668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:38.440668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:38.440668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:38.440668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:38.440668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:38.440668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:38.440668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:38.440668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 08:56:38.440668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:38.444526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 341/660] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:38.445336 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155875b5-6b4e-4d08-a7d8-9411cbc6d9be None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:38.470293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59db190-92b7-4736-902e-4a78fcce26d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:38.470962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59db190-92b7-4736-902e-4a78fcce26d8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:38.471245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59db190-92b7-4736-902e-4a78fcce26d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:38.471516 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59db190-92b7-4736-902e-4a78fcce26d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:38.472279 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59db190-92b7-4736-902e-4a78fcce26d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:38.472580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59db190-92b7-4736-902e-4a78fcce26d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:38.473965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59db190-92b7-4736-902e-4a78fcce26d8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:38.490670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 320/661] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:38.490670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36230f78-8584-4baf-b7e2-6971e3448c4b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:38.490670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36230f78-8584-4baf-b7e2-6971e3448c4b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:38.490670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36230f78-8584-4baf-b7e2-6971e3448c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:38.505594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-36230f78-8584-4baf-b7e2-6971e3448c4b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:38.506808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-36230f78-8584-4baf-b7e2-6971e3448c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:38.508866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:38.552437 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:38.558940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155875b5-6b4e-4d08-a7d8-9411cbc6d9be None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:38.583888 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 321/662] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:38.584860 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36230f78-8584-4baf-b7e2-6971e3448c4b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:38.602545 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d8941-5199-4f6b-8d80-27329238c65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:38.603723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d8941-5199-4f6b-8d80-27329238c65f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:38.604731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d8941-5199-4f6b-8d80-27329238c65f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:38.605536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d8941-5199-4f6b-8d80-27329238c65f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:38.685335 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d8941-5199-4f6b-8d80-27329238c65f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:38.686843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d8941-5199-4f6b-8d80-27329238c65f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:38.702198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36230f78-8584-4baf-b7e2-6971e3448c4b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:38.745609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa87b89-47ab-4d1b-a4e2-2d625f6455ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:38.746291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa87b89-47ab-4d1b-a4e2-2d625f6455ba None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:38.746576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa87b89-47ab-4d1b-a4e2-2d625f6455ba None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:38.746853 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa87b89-47ab-4d1b-a4e2-2d625f6455ba None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:38.747735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa87b89-47ab-4d1b-a4e2-2d625f6455ba None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:38.748024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa87b89-47ab-4d1b-a4e2-2d625f6455ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:38.750274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa87b89-47ab-4d1b-a4e2-2d625f6455ba None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:38.758761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:38.776027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 322/663] 10.222.0.13 () {56 vars in 1139 bytes} [Thu Jul 21 08:56:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:38.776027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa87b89-47ab-4d1b-a4e2-2d625f6455ba None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:38.795493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:38.802685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 342/664] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:38.803534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d8941-5199-4f6b-8d80-27329238c65f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:38.897792 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa87b89-47ab-4d1b-a4e2-2d625f6455ba None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:38.910043 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d8941-5199-4f6b-8d80-27329238c65f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:38.938195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b02eb2-c91e-4e33-bb8a-40bb21bccbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:38.938904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b02eb2-c91e-4e33-bb8a-40bb21bccbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:38.939183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b02eb2-c91e-4e33-bb8a-40bb21bccbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:38.939442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b02eb2-c91e-4e33-bb8a-40bb21bccbb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:38.946558 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b02eb2-c91e-4e33-bb8a-40bb21bccbb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:38.948058 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b02eb2-c91e-4e33-bb8a-40bb21bccbb6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:38.950852 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d1b1e-e236-4cae-9af5-0fc810519bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:38.951900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d1b1e-e236-4cae-9af5-0fc810519bcc None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:38.952425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d1b1e-e236-4cae-9af5-0fc810519bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:38.952948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d1b1e-e236-4cae-9af5-0fc810519bcc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:38.961963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b8b02eb2-c91e-4e33-bb8a-40bb21bccbb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:38.961963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:38.961963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:38.961963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:38.961963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:38.961963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:38.961963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:38.961963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:38.961963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:38.961963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:38.961963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:38.961963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:38.961963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:38.961963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:38.961963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:38.963029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:38.964061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:38.964061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:38.964061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:38.964061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:38.964061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:38.964061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:38.964061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:38.964061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:38.964061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 323/665] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:38.964598 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b02eb2-c91e-4e33-bb8a-40bb21bccbb6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:38.967505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d1b1e-e236-4cae-9af5-0fc810519bcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-897d1b1e-e236-4cae-9af5-0fc810519bcc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:38.976500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:38.977425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:38.978255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:38.978255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:38.978255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:38.978255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:38.978255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:38.978255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:38.978255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:38.978255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:38.978255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:38.978255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:38.978255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:38.978255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:38.979524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 343/666] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:38.980463 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d1b1e-e236-4cae-9af5-0fc810519bcc None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:39.041914 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:39.063101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:39.112116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b02eb2-c91e-4e33-bb8a-40bb21bccbb6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:39.129452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d1b1e-e236-4cae-9af5-0fc810519bcc None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:39.151610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d951a4e-9d3b-4108-a336-73183ae372a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:39.152504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d951a4e-9d3b-4108-a336-73183ae372a8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:39.152636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d951a4e-9d3b-4108-a336-73183ae372a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:39.152843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d951a4e-9d3b-4108-a336-73183ae372a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:39.179770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6ecb3-d702-45f8-bdd1-c9d7e6934ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:39.180589 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6ecb3-d702-45f8-bdd1-c9d7e6934ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:39.181043 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6ecb3-d702-45f8-bdd1-c9d7e6934ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:39.181291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6ecb3-d702-45f8-bdd1-c9d7e6934ba3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:39.195217 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6ecb3-d702-45f8-bdd1-c9d7e6934ba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:39.198499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6ecb3-d702-45f8-bdd1-c9d7e6934ba3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:39.233753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-84f6ecb3-d702-45f8-bdd1-c9d7e6934ba3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:39.233753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:39.233753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:39.233753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:39.233753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:39.233753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:39.233753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:39.233753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:39.233753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:39.233753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:39.233753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:39.233753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:39.233753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:39.233753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:39.233753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:39.235284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:39.236185 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:39.236185 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:39.236185 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:39.236185 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:39.236185 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 344/667] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:39.237554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6ecb3-d702-45f8-bdd1-c9d7e6934ba3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:39.243686 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d951a4e-9d3b-4108-a336-73183ae372a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:39.245236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d951a4e-9d3b-4108-a336-73183ae372a8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:39.309132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:39.357530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6ecb3-d702-45f8-bdd1-c9d7e6934ba3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:39.368126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 324/668] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:39.368126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d951a4e-9d3b-4108-a336-73183ae372a8 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:39.371075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:39.416069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ab564-2585-4e24-8cef-b8cd2da699e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:39.416789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ab564-2585-4e24-8cef-b8cd2da699e8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:39.417079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ab564-2585-4e24-8cef-b8cd2da699e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:39.417385 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ab564-2585-4e24-8cef-b8cd2da699e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:39.418175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ab564-2585-4e24-8cef-b8cd2da699e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:39.418490 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ab564-2585-4e24-8cef-b8cd2da699e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:39.420379 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ab564-2585-4e24-8cef-b8cd2da699e8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:39.439787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 345/669] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:39.440763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ab564-2585-4e24-8cef-b8cd2da699e8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:39.482646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d951a4e-9d3b-4108-a336-73183ae372a8 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:39.525185 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b75050-38e8-4941-83d6-9e0ff8f240b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:39.525858 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b75050-38e8-4941-83d6-9e0ff8f240b0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:39.526145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b75050-38e8-4941-83d6-9e0ff8f240b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:39.526420 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b75050-38e8-4941-83d6-9e0ff8f240b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:39.527180 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b75050-38e8-4941-83d6-9e0ff8f240b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:39.527491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b75050-38e8-4941-83d6-9e0ff8f240b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:39.528925 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b75050-38e8-4941-83d6-9e0ff8f240b0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:39.538504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 325/670] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:39.539361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b75050-38e8-4941-83d6-9e0ff8f240b0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:39.565986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:39.569366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ab564-2585-4e24-8cef-b8cd2da699e8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:39.611560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3602b-ad16-4e28-980f-a66e5d8b2a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:39.612288 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3602b-ad16-4e28-980f-a66e5d8b2a7a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:39.612576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3602b-ad16-4e28-980f-a66e5d8b2a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:39.612856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3602b-ad16-4e28-980f-a66e5d8b2a7a None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:39.614127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3602b-ad16-4e28-980f-a66e5d8b2a7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:39.614444 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3602b-ad16-4e28-980f-a66e5d8b2a7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:39.615872 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3602b-ad16-4e28-980f-a66e5d8b2a7a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:39.631224 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 346/671] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:39.631224 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3602b-ad16-4e28-980f-a66e5d8b2a7a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:39.669727 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:39.676842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b75050-38e8-4941-83d6-9e0ff8f240b0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:39.723185 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80df38d-4184-43dd-b219-c1aa72c488f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:39.724235 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80df38d-4184-43dd-b219-c1aa72c488f6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:39.724675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80df38d-4184-43dd-b219-c1aa72c488f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:39.725183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80df38d-4184-43dd-b219-c1aa72c488f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:39.738750 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80df38d-4184-43dd-b219-c1aa72c488f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:39.740385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80df38d-4184-43dd-b219-c1aa72c488f6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:39.751024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3602b-ad16-4e28-980f-a66e5d8b2a7a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:39.760082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-f80df38d-4184-43dd-b219-c1aa72c488f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:39.760082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:39.760082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:39.760082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:39.760082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:39.760082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:39.760082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:39.760082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:39.760082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:39.760082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:39.760082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:39.760082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:39.760082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:39.760082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:39.760082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:39.761120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:39.761981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:39.761981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:39.761981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:39.761981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:39.761981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 326/672] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:39.762334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80df38d-4184-43dd-b219-c1aa72c488f6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:39.796118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0791d-e645-43dc-9a7e-1516a91ab179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:39.797108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0791d-e645-43dc-9a7e-1516a91ab179 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:39.797532 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0791d-e645-43dc-9a7e-1516a91ab179 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:39.797946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0791d-e645-43dc-9a7e-1516a91ab179 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:39.799080 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0791d-e645-43dc-9a7e-1516a91ab179 None admin] RBAC: Adding query filter params (role.id=8511cf3e5d2c4acdae6d0ffda5e72c63, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=e053b6b25a4e4a20a05e6ffa6683222b) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:39.799674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0791d-e645-43dc-9a7e-1516a91ab179 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:39.801325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0791d-e645-43dc-9a7e-1516a91ab179 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:39.809287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 347/673] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:56:39 2022] GET /identity/v3/role_assignments?user.id=e053b6b25a4e4a20a05e6ffa6683222b&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:39.810697 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0791d-e645-43dc-9a7e-1516a91ab179 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:39.874720 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:39.915466 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80df38d-4184-43dd-b219-c1aa72c488f6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:39.955705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:39.958519 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be9d9e-143a-4257-b32b-6b1546b9302e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:39.959677 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be9d9e-143a-4257-b32b-6b1546b9302e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:39.960170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be9d9e-143a-4257-b32b-6b1546b9302e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:39.960634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be9d9e-143a-4257-b32b-6b1546b9302e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:39.971113 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0791d-e645-43dc-9a7e-1516a91ab179 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:39.978170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be9d9e-143a-4257-b32b-6b1546b9302e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:39.979709 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be9d9e-143a-4257-b32b-6b1546b9302e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:39.999847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b6be9d9e-143a-4257-b32b-6b1546b9302e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:39.999847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:39.999847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:39.999847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:39.999847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:39.999847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:39.999847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:39.999847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:39.999847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:39.999847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:39.999847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:39.999847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:39.999847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:39.999847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:39.999847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:40.000787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:40.001704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:40.001704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:40.001704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:40.001704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:40.001704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 327/674] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:40.002295 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be9d9e-143a-4257-b32b-6b1546b9302e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:40.017530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c17b5-7c09-4184-b64d-5e1e9ae5f05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:40.018438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c17b5-7c09-4184-b64d-5e1e9ae5f05d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:40.018791 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c17b5-7c09-4184-b64d-5e1e9ae5f05d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:40.019135 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c17b5-7c09-4184-b64d-5e1e9ae5f05d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:40.028011 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c17b5-7c09-4184-b64d-5e1e9ae5f05d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:40.029776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c17b5-7c09-4184-b64d-5e1e9ae5f05d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:40.046457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-878c17b5-7c09-4184-b64d-5e1e9ae5f05d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:40.046457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:40.046457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:40.046457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:40.046457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:40.046457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:40.046457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:40.046457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:40.046457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:40.046457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:40.046457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:40.046457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:40.046457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:40.046457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:40.046457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:40.047389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:40.048291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:40.048291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:40.048291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:40.048291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:40.048291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:40.048291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:40.048291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:40.048291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:40.048716 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 348/675] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:40.049849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c17b5-7c09-4184-b64d-5e1e9ae5f05d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:40.120691 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be9d9e-143a-4257-b32b-6b1546b9302e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:40.147222 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:40.160725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f41fa7-e9d5-4654-9b62-9d1fcce46260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:40.161462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f41fa7-e9d5-4654-9b62-9d1fcce46260 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:40.161678 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f41fa7-e9d5-4654-9b62-9d1fcce46260 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:40.161958 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f41fa7-e9d5-4654-9b62-9d1fcce46260 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:40.162692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f41fa7-e9d5-4654-9b62-9d1fcce46260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:40.162999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f41fa7-e9d5-4654-9b62-9d1fcce46260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:40.164442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f41fa7-e9d5-4654-9b62-9d1fcce46260 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:40.176627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c17b5-7c09-4184-b64d-5e1e9ae5f05d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:40.177418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 328/676] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:40.178801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241859d4-82ae-4e84-8db0-fa621e2bf6e3 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:40.179094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241859d4-82ae-4e84-8db0-fa621e2bf6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:40.179370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241859d4-82ae-4e84-8db0-fa621e2bf6e3 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:40.180784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 329/677] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:40.185530 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241859d4-82ae-4e84-8db0-fa621e2bf6e3 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:40.218615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd7d4f-af9d-4c5a-9d9d-79152380272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:40.219278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd7d4f-af9d-4c5a-9d9d-79152380272f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:40.219564 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd7d4f-af9d-4c5a-9d9d-79152380272f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:40.219847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd7d4f-af9d-4c5a-9d9d-79152380272f None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:40.220608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd7d4f-af9d-4c5a-9d9d-79152380272f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:40.220917 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd7d4f-af9d-4c5a-9d9d-79152380272f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:40.222303 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd7d4f-af9d-4c5a-9d9d-79152380272f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:40.236302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 349/678] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:40.237230 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd7d4f-af9d-4c5a-9d9d-79152380272f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:40.254883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:40.300086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241859d4-82ae-4e84-8db0-fa621e2bf6e3 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:40.335183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a098ee-75b5-4f02-a722-b6b3517f4761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:40.335873 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a098ee-75b5-4f02-a722-b6b3517f4761 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:40.336284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a098ee-75b5-4f02-a722-b6b3517f4761 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:40.336567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a098ee-75b5-4f02-a722-b6b3517f4761 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:40.337355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a098ee-75b5-4f02-a722-b6b3517f4761 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:40.337664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a098ee-75b5-4f02-a722-b6b3517f4761 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:40.339114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a098ee-75b5-4f02-a722-b6b3517f4761 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:40.355464 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 330/679] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:40.355678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd7d4f-af9d-4c5a-9d9d-79152380272f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:40.355777 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15675d7d-0700-43b3-98c1-457405576e67 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:40.355777 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15675d7d-0700-43b3-98c1-457405576e67 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:40.355777 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15675d7d-0700-43b3-98c1-457405576e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:40.372203 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-15675d7d-0700-43b3-98c1-457405576e67 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:40.374310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-15675d7d-0700-43b3-98c1-457405576e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:40.394214 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d24f6-9d2c-4b64-99b7-e312ea04778c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:40.395003 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d24f6-9d2c-4b64-99b7-e312ea04778c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:40.395291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d24f6-9d2c-4b64-99b7-e312ea04778c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:40.395562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d24f6-9d2c-4b64-99b7-e312ea04778c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:40.403238 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d24f6-9d2c-4b64-99b7-e312ea04778c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:40.405267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d24f6-9d2c-4b64-99b7-e312ea04778c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:40.421255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-ec6d24f6-9d2c-4b64-99b7-e312ea04778c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:40.421255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:40.421255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:40.421255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:40.421255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:40.421255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:40.421255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:40.421255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:40.421255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:40.421255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:40.421255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:40.421255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:40.421255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:40.421255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:40.421255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:40.422195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:40.423683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:40.423683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:40.423683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:40.423683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:40.423683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:40.423683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:40.423683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:40.423683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:40.423683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 350/680] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:40.423683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d24f6-9d2c-4b64-99b7-e312ea04778c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:40.433404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:40.467974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 331/681] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:40.468819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15675d7d-0700-43b3-98c1-457405576e67 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:40.512373 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:40.543629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d24f6-9d2c-4b64-99b7-e312ea04778c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:40.575657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15675d7d-0700-43b3-98c1-457405576e67 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:40.581248 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebaf64-1473-4119-a7c5-f36cb4348b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:40.582147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebaf64-1473-4119-a7c5-f36cb4348b54 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:40.582507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebaf64-1473-4119-a7c5-f36cb4348b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:40.582860 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebaf64-1473-4119-a7c5-f36cb4348b54 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:40.590583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ebaf64-1473-4119-a7c5-f36cb4348b54 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:40.592828 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ebaf64-1473-4119-a7c5-f36cb4348b54 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-d1ebaf64-1473-4119-a7c5-f36cb4348b54 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:40.607525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:40.609325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:40.610273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:40.610273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:40.610273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:40.610273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:40.610273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:40.610273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:40.610273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:40.610273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:40.610273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:56:40.610273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:40.610273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 351/682] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:56:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:40.611504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebaf64-1473-4119-a7c5-f36cb4348b54 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:40.617344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb67e8-a3ef-4ff0-a346-846446b138bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:40.618059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb67e8-a3ef-4ff0-a346-846446b138bd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:40.618347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb67e8-a3ef-4ff0-a346-846446b138bd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:40.618626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb67e8-a3ef-4ff0-a346-846446b138bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:40.619467 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb67e8-a3ef-4ff0-a346-846446b138bd None admin] RBAC: Adding query filter params (role.id=8511cf3e5d2c4acdae6d0ffda5e72c63, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=8a3b3421e9d8427388f4fd2be5a601cd) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:40.619776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb67e8-a3ef-4ff0-a346-846446b138bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:40.621292 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb67e8-a3ef-4ff0-a346-846446b138bd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:40.631327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 332/683] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:56:40 2022] GET /identity/v3/role_assignments?user.id=8a3b3421e9d8427388f4fd2be5a601cd&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:40.631327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb67e8-a3ef-4ff0-a346-846446b138bd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:40.673439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:40.770877 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb67e8-a3ef-4ff0-a346-846446b138bd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:40.775635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebaf64-1473-4119-a7c5-f36cb4348b54 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:40.821892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:40.823570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61d8f4-5793-4445-86a4-b7bf780dd5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:40.824256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61d8f4-5793-4445-86a4-b7bf780dd5fc None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:40.824598 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61d8f4-5793-4445-86a4-b7bf780dd5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:40.824979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61d8f4-5793-4445-86a4-b7bf780dd5fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:40.829057 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebddbbe-cd27-4235-816e-fb3cd66914a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:40.833052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebddbbe-cd27-4235-816e-fb3cd66914a8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:40.833052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebddbbe-cd27-4235-816e-fb3cd66914a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:40.833052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebddbbe-cd27-4235-816e-fb3cd66914a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:40.833052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebddbbe-cd27-4235-816e-fb3cd66914a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:40.833052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebddbbe-cd27-4235-816e-fb3cd66914a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:40.833052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebddbbe-cd27-4235-816e-fb3cd66914a8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:40.842484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61d8f4-5793-4445-86a4-b7bf780dd5fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:40.848695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 352/684] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:40.849876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebddbbe-cd27-4235-816e-fb3cd66914a8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-6e61d8f4-5793-4445-86a4-b7bf780dd5fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:40.850580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:40.851482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:40.852370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:40.852370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:40.852370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:40.852370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:40.852370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:40.852370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:40.852370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:40.852370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:40.852370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:40.852370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:40.852370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:40.852370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:40.852370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 333/685] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:56:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:40.858778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61d8f4-5793-4445-86a4-b7bf780dd5fc None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:40.976471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:40.981707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebddbbe-cd27-4235-816e-fb3cd66914a8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:40.986283 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61d8f4-5793-4445-86a4-b7bf780dd5fc None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:41.023154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03f9cf-1c4e-4aa8-be38-fc6cc72db3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:41.023743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03f9cf-1c4e-4aa8-be38-fc6cc72db3db None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:41.024055 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03f9cf-1c4e-4aa8-be38-fc6cc72db3db None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:41.024333 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03f9cf-1c4e-4aa8-be38-fc6cc72db3db None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:41.025179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03f9cf-1c4e-4aa8-be38-fc6cc72db3db None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:41.025485 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03f9cf-1c4e-4aa8-be38-fc6cc72db3db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:41.027040 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03f9cf-1c4e-4aa8-be38-fc6cc72db3db None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:41.030768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9aa6e2-084b-479c-a53d-345f7455a748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:41.031496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9aa6e2-084b-479c-a53d-345f7455a748 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:41.031849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9aa6e2-084b-479c-a53d-345f7455a748 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:41.032214 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9aa6e2-084b-479c-a53d-345f7455a748 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:41.033116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9aa6e2-084b-479c-a53d-345f7455a748 None admin] RBAC: Adding query filter params (role.id=8511cf3e5d2c4acdae6d0ffda5e72c63, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=9e2a13c2cb864175a082de3ca8b166c9) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:41.033500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9aa6e2-084b-479c-a53d-345f7455a748 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:41.035024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9aa6e2-084b-479c-a53d-345f7455a748 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:41.041939 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 334/686] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:56:40 2022] GET /identity/v3/role_assignments?user.id=9e2a13c2cb864175a082de3ca8b166c9&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:41.042864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9aa6e2-084b-479c-a53d-345f7455a748 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:41.055648 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 353/687] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jul 21 08:56:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:41.061607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03f9cf-1c4e-4aa8-be38-fc6cc72db3db None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:41.176432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:41.260240 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03f9cf-1c4e-4aa8-be38-fc6cc72db3db None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:41.260869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9aa6e2-084b-479c-a53d-345f7455a748 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:41.303392 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5c093-c738-488e-b436-00cb2b641b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:41.304031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5c093-c738-488e-b436-00cb2b641b27 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:41.304318 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5c093-c738-488e-b436-00cb2b641b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:41.304596 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5c093-c738-488e-b436-00cb2b641b27 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:41.305543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5c093-c738-488e-b436-00cb2b641b27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:41.306696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5c093-c738-488e-b436-00cb2b641b27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:41.308136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5c093-c738-488e-b436-00cb2b641b27 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:41.308298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39189e64-458c-429e-9131-0e6a2a12f329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:41.309006 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39189e64-458c-429e-9131-0e6a2a12f329 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:41.309360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39189e64-458c-429e-9131-0e6a2a12f329 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:41.309704 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39189e64-458c-429e-9131-0e6a2a12f329 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:41.325473 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 335/688] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:41.327186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5c093-c738-488e-b436-00cb2b641b27 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:41.357445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:41.402965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39189e64-458c-429e-9131-0e6a2a12f329 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:41.404830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39189e64-458c-429e-9131-0e6a2a12f329 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:41.456049 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5c093-c738-488e-b436-00cb2b641b27 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:41.469187 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:41.497883 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85af980-1123-4a4c-b767-db553a2ec1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:41.501114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85af980-1123-4a4c-b767-db553a2ec1ec None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:41.501596 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85af980-1123-4a4c-b767-db553a2ec1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:41.502040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85af980-1123-4a4c-b767-db553a2ec1ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:41.514920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85af980-1123-4a4c-b767-db553a2ec1ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:41.516783 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85af980-1123-4a4c-b767-db553a2ec1ec None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:41.533441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 354/689] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:41.540729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39189e64-458c-429e-9131-0e6a2a12f329 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:41.552089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-e85af980-1123-4a4c-b767-db553a2ec1ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:41.552089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:41.552089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:41.552089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:41.552089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:41.552089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:41.552089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:41.552089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:41.552089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:41.552089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:41.552089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:41.552089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:41.552089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:41.552089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:41.552089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:41.553183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:41.554301 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:41.554301 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:41.554301 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:41.554301 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:41.562042 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 336/690] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:41.568079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85af980-1123-4a4c-b767-db553a2ec1ec None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:41.712843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:41.751942 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39189e64-458c-429e-9131-0e6a2a12f329 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:41.753551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85af980-1123-4a4c-b767-db553a2ec1ec None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:41.809480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c36b1b-8429-4262-9f83-79680edfe8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:41.810520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c36b1b-8429-4262-9f83-79680edfe8cb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:41.811134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c36b1b-8429-4262-9f83-79680edfe8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:41.811627 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c36b1b-8429-4262-9f83-79680edfe8cb None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:41.813132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c36b1b-8429-4262-9f83-79680edfe8cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:41.815413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c36b1b-8429-4262-9f83-79680edfe8cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:41.816665 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c36b1b-8429-4262-9f83-79680edfe8cb None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:41.817375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ea003-e5ef-465a-9aee-db4c88eec042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:41.818363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ea003-e5ef-465a-9aee-db4c88eec042 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:41.818792 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ea003-e5ef-465a-9aee-db4c88eec042 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:41.819281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ea003-e5ef-465a-9aee-db4c88eec042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:41.845131 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ea003-e5ef-465a-9aee-db4c88eec042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:41.846849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ea003-e5ef-465a-9aee-db4c88eec042 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:41.851758 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 337/691] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:41.855511 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c36b1b-8429-4262-9f83-79680edfe8cb None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:41.899120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-627ea003-e5ef-465a-9aee-db4c88eec042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:41.899120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:41.899120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:41.899120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:41.899120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:41.899120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:41.899120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:41.899120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:41.899120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:41.899120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:41.899120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:41.899120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:41.899120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:41.899120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:41.899120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:41.900159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:41.901101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:41.901101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:41.901101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:41.901101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:41.901319 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 355/692] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:41.910184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ea003-e5ef-465a-9aee-db4c88eec042 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:41.916895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:42.093838 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c36b1b-8429-4262-9f83-79680edfe8cb None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:42.111088 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ea003-e5ef-465a-9aee-db4c88eec042 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:42.133779 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9933e1f-c017-4dab-bba9-5b8352dd80e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:42.134701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9933e1f-c017-4dab-bba9-5b8352dd80e7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:42.135068 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9933e1f-c017-4dab-bba9-5b8352dd80e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:42.135492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9933e1f-c017-4dab-bba9-5b8352dd80e7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:42.137928 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:42.146551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9933e1f-c017-4dab-bba9-5b8352dd80e7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:42.148191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9933e1f-c017-4dab-bba9-5b8352dd80e7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:42.160744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587de6f-bf35-4580-a581-e7dc32e6cb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:42.161413 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587de6f-bf35-4580-a581-e7dc32e6cb16 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:42.161700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587de6f-bf35-4580-a581-e7dc32e6cb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:42.161977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587de6f-bf35-4580-a581-e7dc32e6cb16 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:42.162714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587de6f-bf35-4580-a581-e7dc32e6cb16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:42.163018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587de6f-bf35-4580-a581-e7dc32e6cb16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:42.164473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587de6f-bf35-4580-a581-e7dc32e6cb16 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-c9933e1f-c017-4dab-bba9-5b8352dd80e7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:42.165769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:42.166646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:42.167561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:42.167561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:42.167561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:42.167561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:42.167561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:42.167561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:42.167561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:42.167561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:42.167561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:56:42.167561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:42.167561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 338/693] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jul 21 08:56:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:42.168678 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e136a5-b882-4bed-b1e1-8287f1dc5608 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:42.171938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e136a5-b882-4bed-b1e1-8287f1dc5608 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:42.171938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e136a5-b882-4bed-b1e1-8287f1dc5608 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:42.171938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 339/694] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:42.173610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e136a5-b882-4bed-b1e1-8287f1dc5608 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:42.184794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 356/695] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:42.186854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d78bdf-b559-4cbc-bc31-ae65e3d1131a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:42.187101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d78bdf-b559-4cbc-bc31-ae65e3d1131a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:42.187382 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d78bdf-b559-4cbc-bc31-ae65e3d1131a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:42.210170 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-11d78bdf-b559-4cbc-bc31-ae65e3d1131a None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:42.212296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-11d78bdf-b559-4cbc-bc31-ae65e3d1131a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:42.294212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:42.309228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 357/696] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:42.310186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d78bdf-b559-4cbc-bc31-ae65e3d1131a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:42.317852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e136a5-b882-4bed-b1e1-8287f1dc5608 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:42.375228 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc62449-ec9b-4b2d-93ac-0aebe2dfbebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:42.375969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc62449-ec9b-4b2d-93ac-0aebe2dfbebe None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:42.376286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc62449-ec9b-4b2d-93ac-0aebe2dfbebe None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:42.376567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc62449-ec9b-4b2d-93ac-0aebe2dfbebe None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:42.377533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc62449-ec9b-4b2d-93ac-0aebe2dfbebe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:42.377869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc62449-ec9b-4b2d-93ac-0aebe2dfbebe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:42.379453 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc62449-ec9b-4b2d-93ac-0aebe2dfbebe None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:42.406266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 340/697] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jul 21 08:56:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:42.407312 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc62449-ec9b-4b2d-93ac-0aebe2dfbebe None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:42.445847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d78bdf-b559-4cbc-bc31-ae65e3d1131a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:42.481649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:42.492292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1fd04a-69b3-4e0f-8722-171ad3c0591d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:42.493502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fd04a-69b3-4e0f-8722-171ad3c0591d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:42.494196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fd04a-69b3-4e0f-8722-171ad3c0591d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:42.494653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fd04a-69b3-4e0f-8722-171ad3c0591d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:42.504347 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1fd04a-69b3-4e0f-8722-171ad3c0591d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:42.506379 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1fd04a-69b3-4e0f-8722-171ad3c0591d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:42.525370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-9c1fd04a-69b3-4e0f-8722-171ad3c0591d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:42.525370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:42.525370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:42.525370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:42.525370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:42.525370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:42.525370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:42.525370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:42.525370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:42.525370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:42.525370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:42.525370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:42.525370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:42.525370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:42.525370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:42.526327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:42.527200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:42.527200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:42.527200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:42.527200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:42.527200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:42.527200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:42.527200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:42.527200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:42.535093 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 358/698] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:42.536496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1fd04a-69b3-4e0f-8722-171ad3c0591d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:42.537683 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc62449-ec9b-4b2d-93ac-0aebe2dfbebe None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:42.586125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937028f1-752f-4dcd-9187-0fcf6ec2a014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:42.586949 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937028f1-752f-4dcd-9187-0fcf6ec2a014 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:42.587232 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937028f1-752f-4dcd-9187-0fcf6ec2a014 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:42.587510 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937028f1-752f-4dcd-9187-0fcf6ec2a014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:42.605921 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937028f1-752f-4dcd-9187-0fcf6ec2a014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:42.610742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937028f1-752f-4dcd-9187-0fcf6ec2a014 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:42.612851 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:42.634781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-937028f1-752f-4dcd-9187-0fcf6ec2a014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:42.634781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:42.634781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:42.634781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:42.634781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:42.634781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:42.634781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:42.634781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:42.634781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:42.634781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:42.634781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:42.634781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:42.634781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:42.634781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:42.634781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:42.635698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:42.636609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:42.636609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:42.636609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:42.636609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:42.640010 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 341/699] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:42.641171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937028f1-752f-4dcd-9187-0fcf6ec2a014 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:42.689818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1fd04a-69b3-4e0f-8722-171ad3c0591d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:42.737897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68155c-bf79-4d00-b154-9df0c499e9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:42.738797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68155c-bf79-4d00-b154-9df0c499e9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:42.739003 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68155c-bf79-4d00-b154-9df0c499e9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:42.739519 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68155c-bf79-4d00-b154-9df0c499e9f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:42.790502 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937028f1-752f-4dcd-9187-0fcf6ec2a014 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:42.794183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:42.829349 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68155c-bf79-4d00-b154-9df0c499e9f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:42.831530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68155c-bf79-4d00-b154-9df0c499e9f2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:42.837653 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ea72b-1d87-4f58-a7b9-5abaf3783bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:42.838327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ea72b-1d87-4f58-a7b9-5abaf3783bbd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:42.838609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ea72b-1d87-4f58-a7b9-5abaf3783bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:42.838888 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ea72b-1d87-4f58-a7b9-5abaf3783bbd None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:42.839679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ea72b-1d87-4f58-a7b9-5abaf3783bbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:42.840034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ea72b-1d87-4f58-a7b9-5abaf3783bbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:42.841484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ea72b-1d87-4f58-a7b9-5abaf3783bbd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:42.859749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 342/700] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:42.859749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ea72b-1d87-4f58-a7b9-5abaf3783bbd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:42.910922 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:42.983080 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 359/701] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:42.983080 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68155c-bf79-4d00-b154-9df0c499e9f2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:43.026331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ea72b-1d87-4f58-a7b9-5abaf3783bbd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:43.062644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da57c2-d76f-448f-b996-c51dd53222a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:43.063313 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da57c2-d76f-448f-b996-c51dd53222a9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:43.063601 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da57c2-d76f-448f-b996-c51dd53222a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:43.063878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da57c2-d76f-448f-b996-c51dd53222a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:43.064639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da57c2-d76f-448f-b996-c51dd53222a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:43.064945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da57c2-d76f-448f-b996-c51dd53222a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:43.066361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da57c2-d76f-448f-b996-c51dd53222a9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:43.079147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 343/702] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:43.080031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da57c2-d76f-448f-b996-c51dd53222a9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:43.112044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68155c-bf79-4d00-b154-9df0c499e9f2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:43.113253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:43.151482 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe2345d-cee4-4a0a-8ecc-8fc5dbd6fbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:43.152178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2345d-cee4-4a0a-8ecc-8fc5dbd6fbcb None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:43.152464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2345d-cee4-4a0a-8ecc-8fc5dbd6fbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:43.152811 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2345d-cee4-4a0a-8ecc-8fc5dbd6fbcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:43.154558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe2345d-cee4-4a0a-8ecc-8fc5dbd6fbcb None admin] RBAC: Adding query filter params (role.id=8511cf3e5d2c4acdae6d0ffda5e72c63, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=061def74ec7f42ce9a174e0944bf499f) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:43.156287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe2345d-cee4-4a0a-8ecc-8fc5dbd6fbcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:43.157807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe2345d-cee4-4a0a-8ecc-8fc5dbd6fbcb None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:43.176397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 360/703] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:56:42 2022] GET /identity/v3/role_assignments?user.id=061def74ec7f42ce9a174e0944bf499f&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:43.176397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10effeee-9b2d-4069-9ff2-0f3fda386065 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:43.176397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10effeee-9b2d-4069-9ff2-0f3fda386065 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:43.176397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10effeee-9b2d-4069-9ff2-0f3fda386065 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:43.176397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 361/704] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:43.176397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10effeee-9b2d-4069-9ff2-0f3fda386065 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:43.204962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:43.236286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da57c2-d76f-448f-b996-c51dd53222a9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:43.278810 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d39523-0c83-44bf-b970-06bf75cc642a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:43.279624 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d39523-0c83-44bf-b970-06bf75cc642a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:43.280983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d39523-0c83-44bf-b970-06bf75cc642a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:43.281265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d39523-0c83-44bf-b970-06bf75cc642a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:43.293999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d39523-0c83-44bf-b970-06bf75cc642a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:43.295880 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d39523-0c83-44bf-b970-06bf75cc642a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:43.317277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-14d39523-0c83-44bf-b970-06bf75cc642a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:43.317277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:43.317277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:43.317277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:43.317277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:43.317277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:43.317277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:43.317277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:43.317277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:43.317277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:43.317277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:43.317277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:43.317277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:43.317277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:43.317277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:43.318743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:43.319698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:43.319698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:43.319698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:43.319698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:43.323377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 344/705] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:43.325045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8829f19-0662-4d64-82b8-c6d8f41c8fb0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:43.325332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8829f19-0662-4d64-82b8-c6d8f41c8fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:43.325607 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8829f19-0662-4d64-82b8-c6d8f41c8fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:43.335573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10effeee-9b2d-4069-9ff2-0f3fda386065 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:43.345698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c8829f19-0662-4d64-82b8-c6d8f41c8fb0 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:43.347014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c8829f19-0662-4d64-82b8-c6d8f41c8fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:43.388682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64b4a2-15d6-4d00-bcde-82f65f5712de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:43.389468 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64b4a2-15d6-4d00-bcde-82f65f5712de None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:43.389748 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64b4a2-15d6-4d00-bcde-82f65f5712de None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:43.390132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64b4a2-15d6-4d00-bcde-82f65f5712de None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:43.390863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64b4a2-15d6-4d00-bcde-82f65f5712de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:43.393541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64b4a2-15d6-4d00-bcde-82f65f5712de None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:43.423808 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 362/706] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:56:43 2022] POST /identity/v3/users => generated 254 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:56:43.424758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64b4a2-15d6-4d00-bcde-82f65f5712de None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:43.438318 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 345/707] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:43.461217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8829f19-0662-4d64-82b8-c6d8f41c8fb0 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:43.473118 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:43.536391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:43.581967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64b4a2-15d6-4d00-bcde-82f65f5712de None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:43.612487 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8829f19-0662-4d64-82b8-c6d8f41c8fb0 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:43.623048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897498a-10e8-4011-99fc-60609a95439b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:43.624371 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897498a-10e8-4011-99fc-60609a95439b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:43.624652 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897498a-10e8-4011-99fc-60609a95439b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:43.625154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897498a-10e8-4011-99fc-60609a95439b None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:43.625917 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897498a-10e8-4011-99fc-60609a95439b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:43.626217 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897498a-10e8-4011-99fc-60609a95439b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:43.627615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897498a-10e8-4011-99fc-60609a95439b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:43.645541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 363/708] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:43.647164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd299ed-15fc-43fa-9462-281c2e5053f8 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:43.647531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd299ed-15fc-43fa-9462-281c2e5053f8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:43.647741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd299ed-15fc-43fa-9462-281c2e5053f8 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:43.649543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 364/709] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:43.650804 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd299ed-15fc-43fa-9462-281c2e5053f8 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:43.655630 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428f134-c5f8-4342-827f-2fcc0c861bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:43.656617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f134-c5f8-4342-827f-2fcc0c861bff None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:43.656890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f134-c5f8-4342-827f-2fcc0c861bff None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:43.657162 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f134-c5f8-4342-827f-2fcc0c861bff None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:43.657825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428f134-c5f8-4342-827f-2fcc0c861bff None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:43.659340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428f134-c5f8-4342-827f-2fcc0c861bff None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b428f134-c5f8-4342-827f-2fcc0c861bff None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:43.675985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:43.676999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:43.677841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 346/710] 10.222.0.13 () {56 vars in 1129 bytes} [Thu Jul 21 08:56:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:43.679200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef414e2-d936-4b79-b7dc-226719fd5405 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:43.679200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef414e2-d936-4b79-b7dc-226719fd5405 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:43.679356 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef414e2-d936-4b79-b7dc-226719fd5405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:43.697480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1ef414e2-d936-4b79-b7dc-226719fd5405 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:43.698602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1ef414e2-d936-4b79-b7dc-226719fd5405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:43.783050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:43.784106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd299ed-15fc-43fa-9462-281c2e5053f8 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:43.788649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 347/711] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:43.789518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef414e2-d936-4b79-b7dc-226719fd5405 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:43.840858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:43.854552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31159124-f2f9-4b6d-b42b-100ccb4ac11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:43.855460 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31159124-f2f9-4b6d-b42b-100ccb4ac11f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:43.855749 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31159124-f2f9-4b6d-b42b-100ccb4ac11f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:43.856057 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31159124-f2f9-4b6d-b42b-100ccb4ac11f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:43.865461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31159124-f2f9-4b6d-b42b-100ccb4ac11f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:43.866966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31159124-f2f9-4b6d-b42b-100ccb4ac11f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:43.883210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-31159124-f2f9-4b6d-b42b-100ccb4ac11f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:43.883210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:43.883210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:43.883210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:43.883210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:43.883210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:43.883210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:43.883210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:43.883210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:43.883210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:43.883210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:43.883210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:43.883210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:43.883210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:43.883210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:43.884192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:43.885488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:43.885488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:43.885488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:43.885488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:43.885488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:43.885488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:43.885488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:43.885488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:43.885488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 365/712] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:43.885959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31159124-f2f9-4b6d-b42b-100ccb4ac11f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:43.906763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef414e2-d936-4b79-b7dc-226719fd5405 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:43.944739 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f3d79-2157-499f-93cf-6ad8327fc935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:43.945418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f3d79-2157-499f-93cf-6ad8327fc935 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:43.945702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f3d79-2157-499f-93cf-6ad8327fc935 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:43.945980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f3d79-2157-499f-93cf-6ad8327fc935 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:43.946698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f3d79-2157-499f-93cf-6ad8327fc935 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:43.947005 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f3d79-2157-499f-93cf-6ad8327fc935 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:43.948506 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f3d79-2157-499f-93cf-6ad8327fc935 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:43.954213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 348/713] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jul 21 08:56:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:43.955039 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f3d79-2157-499f-93cf-6ad8327fc935 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:44.004380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31159124-f2f9-4b6d-b42b-100ccb4ac11f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:44.021467 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:44.046959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc24bc-7ce1-4c28-ae42-d7d79662a8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:44.047866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc24bc-7ce1-4c28-ae42-d7d79662a8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:44.048193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc24bc-7ce1-4c28-ae42-d7d79662a8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:44.048470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc24bc-7ce1-4c28-ae42-d7d79662a8f0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:44.049204 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc24bc-7ce1-4c28-ae42-d7d79662a8f0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:44.050590 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc24bc-7ce1-4c28-ae42-d7d79662a8f0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:44.063800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f3d79-2157-499f-93cf-6ad8327fc935 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-34dc24bc-7ce1-4c28-ae42-d7d79662a8f0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:44.066999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:44.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:44.071229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:44.071229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:44.071229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:44.071229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:44.071229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:44.071229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:44.071229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:44.071229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:44.071229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:44.071229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:44.071229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:56:44.071229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:56:44.071229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:56:44.071229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:44.080391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 366/714] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:43 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:44.080391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc24bc-7ce1-4c28-ae42-d7d79662a8f0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:44.101361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49379d99-80b4-4fdf-8996-6de0158b9345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:44.102034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49379d99-80b4-4fdf-8996-6de0158b9345 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:44.102321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49379d99-80b4-4fdf-8996-6de0158b9345 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:44.102597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49379d99-80b4-4fdf-8996-6de0158b9345 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:44.103390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49379d99-80b4-4fdf-8996-6de0158b9345 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:44.103704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49379d99-80b4-4fdf-8996-6de0158b9345 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:44.105209 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49379d99-80b4-4fdf-8996-6de0158b9345 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:44.117699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 349/715] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:44.118330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49379d99-80b4-4fdf-8996-6de0158b9345 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:44.144173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:44.198341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc24bc-7ce1-4c28-ae42-d7d79662a8f0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:44.230039 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49379d99-80b4-4fdf-8996-6de0158b9345 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:44.239769 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb0894-0943-4864-ae20-0dcc9b3298d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:44.240581 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb0894-0943-4864-ae20-0dcc9b3298d5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:44.240868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb0894-0943-4864-ae20-0dcc9b3298d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:44.241148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb0894-0943-4864-ae20-0dcc9b3298d5 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:44.241973 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb0894-0943-4864-ae20-0dcc9b3298d5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:44.242285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb0894-0943-4864-ae20-0dcc9b3298d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:44.243762 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb0894-0943-4864-ae20-0dcc9b3298d5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:44.252005 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 367/716] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jul 21 08:56:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:44.253021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb0894-0943-4864-ae20-0dcc9b3298d5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:44.256524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:44.269648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ba924-c90c-40f4-a9e6-9868f50a1435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:44.270506 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ba924-c90c-40f4-a9e6-9868f50a1435 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:44.270894 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ba924-c90c-40f4-a9e6-9868f50a1435 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:44.271277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ba924-c90c-40f4-a9e6-9868f50a1435 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:44.272156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ba924-c90c-40f4-a9e6-9868f50a1435 None admin] RBAC: Adding query filter params (name=network) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:44.272567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ba924-c90c-40f4-a9e6-9868f50a1435 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:44.274235 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ba924-c90c-40f4-a9e6-9868f50a1435 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:44.279602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 350/717] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jul 21 08:56:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:44.288371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ba924-c90c-40f4-a9e6-9868f50a1435 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:44.484610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb0894-0943-4864-ae20-0dcc9b3298d5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:44.496323 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ba924-c90c-40f4-a9e6-9868f50a1435 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:44.509260 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:44.525530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a9b9be-6945-4ca9-87f6-5fe2392bb041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:44.526191 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9b9be-6945-4ca9-87f6-5fe2392bb041 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:44.526480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9b9be-6945-4ca9-87f6-5fe2392bb041 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:44.526758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9b9be-6945-4ca9-87f6-5fe2392bb041 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:44.527597 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a9b9be-6945-4ca9-87f6-5fe2392bb041 None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=c48d41dcf9b344a8b2fec77f4dd75fcb) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:44.527924 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a9b9be-6945-4ca9-87f6-5fe2392bb041 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:44.529399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a9b9be-6945-4ca9-87f6-5fe2392bb041 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:44.537290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 368/718] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:56:44 2022] GET /identity/v3/role_assignments?user.id=c48d41dcf9b344a8b2fec77f4dd75fcb&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=000e942517f2469ba36d2e65978bc314 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:44.549002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78953173-0b10-4893-a091-c2f9d9b079d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:44.550087 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78953173-0b10-4893-a091-c2f9d9b079d5 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:44.550587 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78953173-0b10-4893-a091-c2f9d9b079d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:44.551076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78953173-0b10-4893-a091-c2f9d9b079d5 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:44.552116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78953173-0b10-4893-a091-c2f9d9b079d5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:44.552655 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78953173-0b10-4893-a091-c2f9d9b079d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:44.554517 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78953173-0b10-4893-a091-c2f9d9b079d5 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:44.565996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 351/719] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jul 21 08:56:44 2022] GET /identity/v3/services?type=network => generated 124 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:45.671483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84d2e8-98d6-45a5-a524-7d30fdd63227 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:45.673120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84d2e8-98d6-45a5-a524-7d30fdd63227 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:45.673863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84d2e8-98d6-45a5-a524-7d30fdd63227 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:45.676751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 369/720] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:45.683671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4a267-b343-42a9-b8a3-2fc9861892ef None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:45.684999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4a267-b343-42a9-b8a3-2fc9861892ef None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:45.685819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4a267-b343-42a9-b8a3-2fc9861892ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:45.727389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-68a4a267-b343-42a9-b8a3-2fc9861892ef None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:45.729537 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-68a4a267-b343-42a9-b8a3-2fc9861892ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:45.822963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:45.855446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 352/721] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:45.891348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84d2e8-98d6-45a5-a524-7d30fdd63227 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:45.959840 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668aeb0-f5d1-4e67-bdde-4961eb6495e8 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:45.960496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668aeb0-f5d1-4e67-bdde-4961eb6495e8 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:45.961628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668aeb0-f5d1-4e67-bdde-4961eb6495e8 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:45.963488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 353/722] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:56:45.971061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f868929-d140-4a6e-be61-2af1dda9ce8b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:45.971348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f868929-d140-4a6e-be61-2af1dda9ce8b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:45.971627 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f868929-d140-4a6e-be61-2af1dda9ce8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:46.005005 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0f868929-d140-4a6e-be61-2af1dda9ce8b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:46.006207 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0f868929-d140-4a6e-be61-2af1dda9ce8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:46.060402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84d2e8-98d6-45a5-a524-7d30fdd63227 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:46.123463 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38aede-6cd8-421d-bab4-fc67b1c1cda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:46.125944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38aede-6cd8-421d-bab4-fc67b1c1cda5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:46.127149 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38aede-6cd8-421d-bab4-fc67b1c1cda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:46.127799 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38aede-6cd8-421d-bab4-fc67b1c1cda5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:46.140817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 354/723] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:46.146891 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e38aede-6cd8-421d-bab4-fc67b1c1cda5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-3e38aede-6cd8-421d-bab4-fc67b1c1cda5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:46.155083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:46.156122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:46.156954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:46.156954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:46.156954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:46.156954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:46.156954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:46.156954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:46.156954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:46.156954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:46.156954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:46.156954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:46.156954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:46.156954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:46.159777 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 370/724] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:56:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:46.165641 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f868929-d140-4a6e-be61-2af1dda9ce8b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:46.185222 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38aede-6cd8-421d-bab4-fc67b1c1cda5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:46.256382 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:46.379833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38aede-6cd8-421d-bab4-fc67b1c1cda5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:46.387328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f868929-d140-4a6e-be61-2af1dda9ce8b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:46.428101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:46.455788 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcc831-6352-43c1-9ba2-0e48261c187f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:46.457532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcc831-6352-43c1-9ba2-0e48261c187f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:46.459818 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcc831-6352-43c1-9ba2-0e48261c187f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:46.460497 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcc831-6352-43c1-9ba2-0e48261c187f None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:46.461432 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcc831-6352-43c1-9ba2-0e48261c187f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:46.462070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcc831-6352-43c1-9ba2-0e48261c187f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:46.464036 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcc831-6352-43c1-9ba2-0e48261c187f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:46.473719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 355/725] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:46.479495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcc831-6352-43c1-9ba2-0e48261c187f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:46.485286 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96db78-13aa-49aa-81be-a0b8563457fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:46.486100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96db78-13aa-49aa-81be-a0b8563457fd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:46.486390 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96db78-13aa-49aa-81be-a0b8563457fd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:46.486671 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96db78-13aa-49aa-81be-a0b8563457fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:46.528995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96db78-13aa-49aa-81be-a0b8563457fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-8e96db78-13aa-49aa-81be-a0b8563457fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:46.544937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:46.546363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:46.547193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:46.547193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:46.547193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:46.547193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:46.547193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:46.547193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:46.547193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:46.547193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:46.547193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:46.547193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:46.547193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:46.547193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:46.547193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 371/726] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:46.552922 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96db78-13aa-49aa-81be-a0b8563457fd None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:46.593391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcc831-6352-43c1-9ba2-0e48261c187f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:46.632977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1778a-ea7d-4060-92e9-b54bc70aac7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:46.634099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1778a-ea7d-4060-92e9-b54bc70aac7a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:46.634531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1778a-ea7d-4060-92e9-b54bc70aac7a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:46.634948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1778a-ea7d-4060-92e9-b54bc70aac7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:46.648188 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1778a-ea7d-4060-92e9-b54bc70aac7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:46.650243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1778a-ea7d-4060-92e9-b54bc70aac7a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:46.670439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-25c1778a-ea7d-4060-92e9-b54bc70aac7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:46.670439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:46.670439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:46.670439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:46.670439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:46.670439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:46.670439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:46.670439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:46.670439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:46.670439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:46.670439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:46.670439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:46.670439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:46.670439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:46.670439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:46.671443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:46.672443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:46.672443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:46.672443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:46.672443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:46.674366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 356/727] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:46.680526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1778a-ea7d-4060-92e9-b54bc70aac7a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:46.715362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:46.798371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1778a-ea7d-4060-92e9-b54bc70aac7a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:46.845076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db16d8-2dc7-4aef-a624-843237bb5a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:46.846005 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db16d8-2dc7-4aef-a624-843237bb5a06 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:46.846466 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db16d8-2dc7-4aef-a624-843237bb5a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:46.846889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db16d8-2dc7-4aef-a624-843237bb5a06 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:46.848621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db16d8-2dc7-4aef-a624-843237bb5a06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:46.850808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db16d8-2dc7-4aef-a624-843237bb5a06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:46.852403 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db16d8-2dc7-4aef-a624-843237bb5a06 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:46.878736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 357/728] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:46.886824 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96db78-13aa-49aa-81be-a0b8563457fd None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:46.888406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db16d8-2dc7-4aef-a624-843237bb5a06 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:46.907773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:46.982196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f41a6-4a23-4715-92d0-559af6bbe8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:46.983461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f41a6-4a23-4715-92d0-559af6bbe8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:46.984284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f41a6-4a23-4715-92d0-559af6bbe8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:46.984778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f41a6-4a23-4715-92d0-559af6bbe8d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:46.985778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f41a6-4a23-4715-92d0-559af6bbe8d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:46.986292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f41a6-4a23-4715-92d0-559af6bbe8d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:46.987940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f41a6-4a23-4715-92d0-559af6bbe8d5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:46.997726 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 372/729] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:46.999548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36169738-3d52-4256-b78b-72c92d3e970b None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:47.000130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36169738-3d52-4256-b78b-72c92d3e970b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:47.000616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36169738-3d52-4256-b78b-72c92d3e970b None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:47.006730 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 373/730] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:47.007993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36169738-3d52-4256-b78b-72c92d3e970b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:47.016308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:47.020822 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db16d8-2dc7-4aef-a624-843237bb5a06 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:47.066615 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9271d-276b-4ff4-bc46-9d8696528450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:47.067331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9271d-276b-4ff4-bc46-9d8696528450 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:47.067613 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9271d-276b-4ff4-bc46-9d8696528450 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:47.067887 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9271d-276b-4ff4-bc46-9d8696528450 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:47.077981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9271d-276b-4ff4-bc46-9d8696528450 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:47.079543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9271d-276b-4ff4-bc46-9d8696528450 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-01a9271d-276b-4ff4-bc46-9d8696528450 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:47.098643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:47.099725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:47.100731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:47.100731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:47.100731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:47.100731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:47.100731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:47.100731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:47.100731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:47.100731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:47.100731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:56:47.100731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:47.100731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 358/731] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:56:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:47.101830 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fb963-4af3-458e-99c1-6dbbdb9ef0a5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:47.101830 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fb963-4af3-458e-99c1-6dbbdb9ef0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:47.101983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fb963-4af3-458e-99c1-6dbbdb9ef0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:47.123292 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f30fb963-4af3-458e-99c1-6dbbdb9ef0a5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:47.125395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f30fb963-4af3-458e-99c1-6dbbdb9ef0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:47.148642 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36169738-3d52-4256-b78b-72c92d3e970b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:47.184866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23862e-282e-478e-b73f-d11cee34b4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:47.185661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23862e-282e-478e-b73f-d11cee34b4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:47.186004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23862e-282e-478e-b73f-d11cee34b4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:47.186496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23862e-282e-478e-b73f-d11cee34b4a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:47.198141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23862e-282e-478e-b73f-d11cee34b4a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:47.199591 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23862e-282e-478e-b73f-d11cee34b4a4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:47.202994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 359/732] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:47.203841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30fb963-4af3-458e-99c1-6dbbdb9ef0a5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:47.217287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-ae23862e-282e-478e-b73f-d11cee34b4a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:47.217287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:47.217287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:47.217287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:47.217287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:47.217287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:47.217287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:47.217287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:47.217287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:47.217287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:47.217287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:47.217287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:47.217287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:47.217287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:47.217287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:47.218683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:47.222391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:47.222391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:47.222391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:47.222391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:47.222391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 374/733] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:47.222391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97beb92a-dfbb-49fc-bb55-9e0c4d77e29b None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:47.222391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97beb92a-dfbb-49fc-bb55-9e0c4d77e29b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:47.222391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97beb92a-dfbb-49fc-bb55-9e0c4d77e29b None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:47.223700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 375/734] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:47.225011 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c19bd-9d8d-40bc-a8ff-c1b60792fe91 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:47.225507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c19bd-9d8d-40bc-a8ff-c1b60792fe91 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:47.225627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c19bd-9d8d-40bc-a8ff-c1b60792fe91 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:47.226981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 376/735] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:47.228067 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c19bd-9d8d-40bc-a8ff-c1b60792fe91 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:47.248195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:47.293655 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:47.318007 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30fb963-4af3-458e-99c1-6dbbdb9ef0a5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:47.337216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c19bd-9d8d-40bc-a8ff-c1b60792fe91 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:47.353178 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5e5b8-4d8b-4f3c-93e7-fc432a398888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:47.353847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5e5b8-4d8b-4f3c-93e7-fc432a398888 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:47.354132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5e5b8-4d8b-4f3c-93e7-fc432a398888 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:47.354407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5e5b8-4d8b-4f3c-93e7-fc432a398888 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:47.355240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5e5b8-4d8b-4f3c-93e7-fc432a398888 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:47.355547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5e5b8-4d8b-4f3c-93e7-fc432a398888 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:47.357029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5e5b8-4d8b-4f3c-93e7-fc432a398888 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:47.371869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be2d31-f035-4587-b721-44aedbe149ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:47.372551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be2d31-f035-4587-b721-44aedbe149ba None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:47.372835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be2d31-f035-4587-b721-44aedbe149ba None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:47.373103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be2d31-f035-4587-b721-44aedbe149ba None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:47.374425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be2d31-f035-4587-b721-44aedbe149ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:47.374727 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be2d31-f035-4587-b721-44aedbe149ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:47.375612 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 360/736] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jul 21 08:56:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:47.376159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be2d31-f035-4587-b721-44aedbe149ba None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:47.376928 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df90027-a20e-4eb2-a5ca-e046529af214 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:47.377221 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df90027-a20e-4eb2-a5ca-e046529af214 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:47.377498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df90027-a20e-4eb2-a5ca-e046529af214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:47.392713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 377/737] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:47.393387 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be2d31-f035-4587-b721-44aedbe149ba None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:47.394694 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4df90027-a20e-4eb2-a5ca-e046529af214 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:47.396007 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4df90027-a20e-4eb2-a5ca-e046529af214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:47.467472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 361/738] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:47.468812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68080b2-6ad5-477e-b4ff-b12194444760 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:47.469102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68080b2-6ad5-477e-b4ff-b12194444760 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:47.469381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68080b2-6ad5-477e-b4ff-b12194444760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:47.485120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:47.494141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e68080b2-6ad5-477e-b4ff-b12194444760 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:47.495488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e68080b2-6ad5-477e-b4ff-b12194444760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:47.508362 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be2d31-f035-4587-b721-44aedbe149ba None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:47.541819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa29950-4bb1-489b-b4e1-1cb1865c554a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:47.542483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa29950-4bb1-489b-b4e1-1cb1865c554a None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:47.542767 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa29950-4bb1-489b-b4e1-1cb1865c554a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:47.543061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa29950-4bb1-489b-b4e1-1cb1865c554a None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:47.543771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa29950-4bb1-489b-b4e1-1cb1865c554a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:47.545437 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa29950-4bb1-489b-b4e1-1cb1865c554a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:47.555784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:47.560992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 378/739] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jul 21 08:56:47 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:56:47.562472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa29950-4bb1-489b-b4e1-1cb1865c554a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:47.581565 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 362/740] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:56:47.582698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68080b2-6ad5-477e-b4ff-b12194444760 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:47.701692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68080b2-6ad5-477e-b4ff-b12194444760 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:47.702355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa29950-4bb1-489b-b4e1-1cb1865c554a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:47.751155 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60dd81-706f-4abb-a27a-280d3333415b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:47.751822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60dd81-706f-4abb-a27a-280d3333415b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:47.752162 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60dd81-706f-4abb-a27a-280d3333415b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:47.753083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60dd81-706f-4abb-a27a-280d3333415b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:47.753284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ee70c-5560-4af6-bf11-546a0f3abf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:47.754342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ee70c-5560-4af6-bf11-546a0f3abf44 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:47.754646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ee70c-5560-4af6-bf11-546a0f3abf44 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:47.755142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ee70c-5560-4af6-bf11-546a0f3abf44 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:47.762583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:47.763188 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ee70c-5560-4af6-bf11-546a0f3abf44 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:47.764776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ee70c-5560-4af6-bf11-546a0f3abf44 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-a96ee70c-5560-4af6-bf11-546a0f3abf44 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:47.780904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:47.781895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:47.783719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:47.783719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:47.783719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:47.783719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:47.783719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:47.783719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:47.783719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:47.783719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:47.783719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:56:47.783719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:47.783719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 363/741] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jul 21 08:56:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:47.783719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ee70c-5560-4af6-bf11-546a0f3abf44 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:47.840764 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60dd81-706f-4abb-a27a-280d3333415b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:47.842157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60dd81-706f-4abb-a27a-280d3333415b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:47.849120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:47.907861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ee70c-5560-4af6-bf11-546a0f3abf44 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:47.949015 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11920d-e415-4079-af40-0f4f8df4ddcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:47.949686 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11920d-e415-4079-af40-0f4f8df4ddcd None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:47.949969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11920d-e415-4079-af40-0f4f8df4ddcd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:47.950247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11920d-e415-4079-af40-0f4f8df4ddcd None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:47.950969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11920d-e415-4079-af40-0f4f8df4ddcd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:47.953114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc11920d-e415-4079-af40-0f4f8df4ddcd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:47.961808 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 379/742] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:47 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:56:47.961808 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60dd81-706f-4abb-a27a-280d3333415b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:47.977697 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 364/743] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jul 21 08:56:47 2022] POST /identity/v3/services => generated 240 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:56:47.979235 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11920d-e415-4079-af40-0f4f8df4ddcd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:48.010086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:48.095483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60dd81-706f-4abb-a27a-280d3333415b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:48.138010 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11920d-e415-4079-af40-0f4f8df4ddcd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:48.145228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bf9b1-b32b-4223-9906-a67e53a8c75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:48.146218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bf9b1-b32b-4223-9906-a67e53a8c75a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:48.147419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:48.147703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bf9b1-b32b-4223-9906-a67e53a8c75a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:48.148148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bf9b1-b32b-4223-9906-a67e53a8c75a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:48.166688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bf9b1-b32b-4223-9906-a67e53a8c75a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-9f3bf9b1-b32b-4223-9906-a67e53a8c75a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:48.177211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:48.178181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:48.179014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:48.179014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:48.179014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:48.179014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:48.179014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:48.179014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:48.179014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:48.179014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:48.179014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:48.179014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:48.179014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:48.179014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:48.179014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 380/744] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:56:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:48.179687 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bf9b1-b32b-4223-9906-a67e53a8c75a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:48.189243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33798b-95ef-4529-88ef-2161d72003e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:48.190743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33798b-95ef-4529-88ef-2161d72003e4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:48.191041 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33798b-95ef-4529-88ef-2161d72003e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:48.191320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33798b-95ef-4529-88ef-2161d72003e4 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:48.192230 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33798b-95ef-4529-88ef-2161d72003e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:48.192540 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33798b-95ef-4529-88ef-2161d72003e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:48.194712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33798b-95ef-4529-88ef-2161d72003e4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:48.220520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 365/745] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jul 21 08:56:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:48.223193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33798b-95ef-4529-88ef-2161d72003e4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:48.303124 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:48.310235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bf9b1-b32b-4223-9906-a67e53a8c75a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:48.353512 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33798b-95ef-4529-88ef-2161d72003e4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:48.360975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd721d9-611f-4f14-ac32-f5d656c71850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:48.361820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd721d9-611f-4f14-ac32-f5d656c71850 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:48.362260 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd721d9-611f-4f14-ac32-f5d656c71850 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:48.362542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd721d9-611f-4f14-ac32-f5d656c71850 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:48.379866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd721d9-611f-4f14-ac32-f5d656c71850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:48.381683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd721d9-611f-4f14-ac32-f5d656c71850 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:48.395983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027146b-d00d-4209-a562-c3b6bd912753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:48.397047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027146b-d00d-4209-a562-c3b6bd912753 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:48.397535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027146b-d00d-4209-a562-c3b6bd912753 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:48.398024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027146b-d00d-4209-a562-c3b6bd912753 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:48.399118 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027146b-d00d-4209-a562-c3b6bd912753 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:48.399621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027146b-d00d-4209-a562-c3b6bd912753 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:48.401339 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027146b-d00d-4209-a562-c3b6bd912753 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:48.406893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-9fd721d9-611f-4f14-ac32-f5d656c71850 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:48.406893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:48.406893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:48.406893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:48.406893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:48.406893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:48.406893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:48.406893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:48.406893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:48.406893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:48.406893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:48.406893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:48.406893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:48.406893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:48.406893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:48.407843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:48.409058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:48.409058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:48.409058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:48.409058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:48.409288 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 381/746] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:48.410810 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd721d9-611f-4f14-ac32-f5d656c71850 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:48.421716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 366/747] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:48.423080 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027146b-d00d-4209-a562-c3b6bd912753 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:48.455758 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:48.541715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd721d9-611f-4f14-ac32-f5d656c71850 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:48.542409 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027146b-d00d-4209-a562-c3b6bd912753 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:48.579329 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16369d27-435f-45cf-ae57-eee7b63d049e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:48.580436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16369d27-435f-45cf-ae57-eee7b63d049e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:48.581022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16369d27-435f-45cf-ae57-eee7b63d049e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:48.581506 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16369d27-435f-45cf-ae57-eee7b63d049e None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:48.582571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16369d27-435f-45cf-ae57-eee7b63d049e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:48.583126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16369d27-435f-45cf-ae57-eee7b63d049e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:48.584592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79f8b6-ec05-4e37-a5a6-f2025ef94471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:48.585043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16369d27-435f-45cf-ae57-eee7b63d049e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:48.585466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79f8b6-ec05-4e37-a5a6-f2025ef94471 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:48.585848 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79f8b6-ec05-4e37-a5a6-f2025ef94471 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:48.586224 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79f8b6-ec05-4e37-a5a6-f2025ef94471 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:48.599893 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 367/748] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:48.601080 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16369d27-435f-45cf-ae57-eee7b63d049e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:48.609899 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:48.667748 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb79f8b6-ec05-4e37-a5a6-f2025ef94471 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:48.669391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb79f8b6-ec05-4e37-a5a6-f2025ef94471 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:48.707629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:48.730435 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16369d27-435f-45cf-ae57-eee7b63d049e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:48.772923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d2755-9830-4e3e-9492-118b77183401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:48.773716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d2755-9830-4e3e-9492-118b77183401 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:48.773999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d2755-9830-4e3e-9492-118b77183401 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:48.774270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d2755-9830-4e3e-9492-118b77183401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:48.788554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d2755-9830-4e3e-9492-118b77183401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:48.790207 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d2755-9830-4e3e-9492-118b77183401 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:48.792891 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 382/749] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:56:48.793898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79f8b6-ec05-4e37-a5a6-f2025ef94471 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:48.815236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-334d2755-9830-4e3e-9492-118b77183401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:48.815236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:48.815236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:48.815236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:48.815236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:48.815236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:48.815236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:48.815236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:48.815236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:48.815236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:48.815236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:48.815236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:48.815236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:48.815236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:48.815236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:48.816346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:48.817282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:48.817282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:48.817282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:48.817282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:48.821907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 368/750] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:48.822883 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d2755-9830-4e3e-9492-118b77183401 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:48.884435 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:48.912020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79f8b6-ec05-4e37-a5a6-f2025ef94471 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:48.933645 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d2755-9830-4e3e-9492-118b77183401 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:48.949081 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eba1a-67a0-438e-bde2-968725c40496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:48.950216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eba1a-67a0-438e-bde2-968725c40496 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:48.950707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eba1a-67a0-438e-bde2-968725c40496 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:48.951223 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8eba1a-67a0-438e-bde2-968725c40496 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:48.958906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8eba1a-67a0-438e-bde2-968725c40496 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:48.960701 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8eba1a-67a0-438e-bde2-968725c40496 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:48.973204 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b743bda-a6ce-4a92-87cf-6b270b77f415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:48.974004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b743bda-a6ce-4a92-87cf-6b270b77f415 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:48.974284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b743bda-a6ce-4a92-87cf-6b270b77f415 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:48.974559 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b743bda-a6ce-4a92-87cf-6b270b77f415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:48.981563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:48.985855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b743bda-a6ce-4a92-87cf-6b270b77f415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:48.987752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b743bda-a6ce-4a92-87cf-6b270b77f415 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:48.989744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-2e8eba1a-67a0-438e-bde2-968725c40496 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:48.989744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:48.989744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:48.989744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:48.989744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:48.989744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:48.989744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:48.989744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:48.989744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:48.989744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:48.989744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:48.989744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:48.989744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:48.989744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:48.989744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:48.991060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:48.992173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:48.992173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:48.992173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:48.992173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:48.992173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:48.992173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:48.992173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:48.992173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:48.993625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 383/751] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:48.994937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eba1a-67a0-438e-bde2-968725c40496 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:49.011876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-0b743bda-a6ce-4a92-87cf-6b270b77f415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:49.011876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:49.011876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:49.011876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:49.011876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:49.011876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:49.011876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:49.011876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:49.011876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:49.011876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:49.011876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:49.011876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:49.011876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:49.011876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:49.011876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:49.012854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:49.013765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:49.013765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:49.013765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:49.013765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:49.013765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 369/752] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:49.015039 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b743bda-a6ce-4a92-87cf-6b270b77f415 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:49.111593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8eba1a-67a0-438e-bde2-968725c40496 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:49.132338 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:49.136197 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b743bda-a6ce-4a92-87cf-6b270b77f415 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:49.152728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabf767-3855-4072-a87f-89f64ff1e7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:49.153576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabf767-3855-4072-a87f-89f64ff1e7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:49.154109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabf767-3855-4072-a87f-89f64ff1e7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:49.154467 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabf767-3855-4072-a87f-89f64ff1e7b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:49.156810 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabf767-3855-4072-a87f-89f64ff1e7b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:49.157853 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabf767-3855-4072-a87f-89f64ff1e7b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:49.160081 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabf767-3855-4072-a87f-89f64ff1e7b7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:49.178904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb0d77-4082-4c2f-805a-b0031e2e8188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:49.179569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb0d77-4082-4c2f-805a-b0031e2e8188 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:49.179847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb0d77-4082-4c2f-805a-b0031e2e8188 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:49.180146 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb0d77-4082-4c2f-805a-b0031e2e8188 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:49.180934 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb0d77-4082-4c2f-805a-b0031e2e8188 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:49.181130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 384/753] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:49.181237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb0d77-4082-4c2f-805a-b0031e2e8188 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:49.182074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabf767-3855-4072-a87f-89f64ff1e7b7 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:49.182676 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb0d77-4082-4c2f-805a-b0031e2e8188 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:49.206916 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 370/754] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:49.208093 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb0d77-4082-4c2f-805a-b0031e2e8188 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:49.263844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:49.304084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabf767-3855-4072-a87f-89f64ff1e7b7 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:49.321350 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb0d77-4082-4c2f-805a-b0031e2e8188 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:49.341097 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c7462-87de-4bd1-a319-dde5a12d2a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:49.341594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c7462-87de-4bd1-a319-dde5a12d2a41 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:49.341883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c7462-87de-4bd1-a319-dde5a12d2a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:49.342144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c7462-87de-4bd1-a319-dde5a12d2a41 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:49.343064 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c7462-87de-4bd1-a319-dde5a12d2a41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:49.343411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c7462-87de-4bd1-a319-dde5a12d2a41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:49.344902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c7462-87de-4bd1-a319-dde5a12d2a41 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:49.357554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 385/755] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:49.365568 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c7462-87de-4bd1-a319-dde5a12d2a41 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:49.367981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae657d-d59e-4bf1-ae75-8a95dc2a0acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:49.369069 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae657d-d59e-4bf1-ae75-8a95dc2a0acc None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:49.369465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae657d-d59e-4bf1-ae75-8a95dc2a0acc None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:49.369888 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae657d-d59e-4bf1-ae75-8a95dc2a0acc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:49.380592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae657d-d59e-4bf1-ae75-8a95dc2a0acc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:49.382498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae657d-d59e-4bf1-ae75-8a95dc2a0acc None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:49.401605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-c1ae657d-d59e-4bf1-ae75-8a95dc2a0acc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:49.406484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:49.407723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:49.409051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:49.409051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:49.409051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:49.409051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:49.409051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:49.409051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:49.409051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:49.409051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:49.409051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:56:49.409051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:49.413101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 371/756] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jul 21 08:56:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:49.414952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae657d-d59e-4bf1-ae75-8a95dc2a0acc None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:49.485605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c7462-87de-4bd1-a319-dde5a12d2a41 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:49.523001 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75be044-70f9-4bf9-87a7-7b5edff1653c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:49.523837 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75be044-70f9-4bf9-87a7-7b5edff1653c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:49.524241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75be044-70f9-4bf9-87a7-7b5edff1653c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:49.524621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75be044-70f9-4bf9-87a7-7b5edff1653c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:49.525565 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75be044-70f9-4bf9-87a7-7b5edff1653c None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=061def74ec7f42ce9a174e0944bf499f) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:49.525975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75be044-70f9-4bf9-87a7-7b5edff1653c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:49.527615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75be044-70f9-4bf9-87a7-7b5edff1653c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:49.539953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 386/757] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:56:49 2022] GET /identity/v3/role_assignments?user.id=061def74ec7f42ce9a174e0944bf499f&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=000e942517f2469ba36d2e65978bc314 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:49.539953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75be044-70f9-4bf9-87a7-7b5edff1653c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:49.635693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae657d-d59e-4bf1-ae75-8a95dc2a0acc None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:49.665020 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:49.707086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:49.721486 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75be044-70f9-4bf9-87a7-7b5edff1653c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:49.722917 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e13102-358c-4220-859f-dddfce033915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:49.723752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e13102-358c-4220-859f-dddfce033915 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:49.724061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e13102-358c-4220-859f-dddfce033915 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:49.724336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e13102-358c-4220-859f-dddfce033915 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:49.740651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e13102-358c-4220-859f-dddfce033915 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:49.742183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e13102-358c-4220-859f-dddfce033915 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:49.766828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-84e13102-358c-4220-859f-dddfce033915 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:49.766828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:49.766828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:49.766828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:49.766828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:49.766828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:49.766828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:49.766828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:49.766828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:49.766828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:49.766828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:49.766828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:49.766828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:49.766828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:49.766828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:49.767811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:49.768726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:49.768726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:49.768726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:49.768726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:49.768726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:49.768726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:49.768726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:49.768726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:49.772125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 372/758] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:49.786135 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e13102-358c-4220-859f-dddfce033915 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:49.806104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c065d-96e3-4677-9a44-d276bf7ee752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:49.806798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c065d-96e3-4677-9a44-d276bf7ee752 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:49.807086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c065d-96e3-4677-9a44-d276bf7ee752 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:49.807371 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c065d-96e3-4677-9a44-d276bf7ee752 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:49.808275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c065d-96e3-4677-9a44-d276bf7ee752 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:49.808591 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c065d-96e3-4677-9a44-d276bf7ee752 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:49.810048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c065d-96e3-4677-9a44-d276bf7ee752 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:49.845278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 387/759] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jul 21 08:56:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:49.847286 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432fd81-f05e-4a34-9c4b-2a8cd0f00adc None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:49.847778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432fd81-f05e-4a34-9c4b-2a8cd0f00adc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:49.848287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432fd81-f05e-4a34-9c4b-2a8cd0f00adc None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:49.850017 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 388/760] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:49.855080 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432fd81-f05e-4a34-9c4b-2a8cd0f00adc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:49.966127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e13102-358c-4220-859f-dddfce033915 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:50.012568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593b6fa-9557-4bf7-9495-ca4aa55c656d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:50.013673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593b6fa-9557-4bf7-9495-ca4aa55c656d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:50.014171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593b6fa-9557-4bf7-9495-ca4aa55c656d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:50.015153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593b6fa-9557-4bf7-9495-ca4aa55c656d None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:50.016419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593b6fa-9557-4bf7-9495-ca4aa55c656d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:50.016651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432fd81-f05e-4a34-9c4b-2a8cd0f00adc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:50.017481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593b6fa-9557-4bf7-9495-ca4aa55c656d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:50.019516 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593b6fa-9557-4bf7-9495-ca4aa55c656d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:50.035740 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 373/761] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:50.037553 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89ee98-ba6a-4a07-9425-109d2cfefe5b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:50.038065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89ee98-ba6a-4a07-9425-109d2cfefe5b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:50.038551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89ee98-ba6a-4a07-9425-109d2cfefe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:50.062776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ce89ee98-ba6a-4a07-9425-109d2cfefe5b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:50.064061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ce89ee98-ba6a-4a07-9425-109d2cfefe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:50.079138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f48f6-efaf-4c80-8fad-1f6f0846f01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:50.081130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f48f6-efaf-4c80-8fad-1f6f0846f01f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:50.081633 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f48f6-efaf-4c80-8fad-1f6f0846f01f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:50.082121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f48f6-efaf-4c80-8fad-1f6f0846f01f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:50.097880 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:50.099405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f48f6-efaf-4c80-8fad-1f6f0846f01f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:50.101630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f48f6-efaf-4c80-8fad-1f6f0846f01f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:50.138422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-7a1f48f6-efaf-4c80-8fad-1f6f0846f01f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:50.138422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:50.138422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:50.138422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:50.138422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:50.138422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:50.138422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:50.138422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:50.138422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:50.138422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:50.138422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:50.138422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:50.138422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:50.138422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:50.138422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:50.140202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:50.141216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:50.141216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:50.141216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:50.141216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:50.148635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 389/762] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:50.149815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f48f6-efaf-4c80-8fad-1f6f0846f01f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:50.155800 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:50.180135 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 374/763] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:56:50.180135 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89ee98-ba6a-4a07-9425-109d2cfefe5b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:50.288827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f48f6-efaf-4c80-8fad-1f6f0846f01f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:50.310533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89ee98-ba6a-4a07-9425-109d2cfefe5b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:50.335839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee73f5c-8157-40bd-9fd9-57ff5c5f840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:50.336537 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee73f5c-8157-40bd-9fd9-57ff5c5f840f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:50.336823 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee73f5c-8157-40bd-9fd9-57ff5c5f840f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:50.337102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee73f5c-8157-40bd-9fd9-57ff5c5f840f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:50.337943 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee73f5c-8157-40bd-9fd9-57ff5c5f840f None admin] RBAC: Adding query filter params (role.id=8511cf3e5d2c4acdae6d0ffda5e72c63, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=257450db238d4efda783e30cbcd23d4f) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:50.338260 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee73f5c-8157-40bd-9fd9-57ff5c5f840f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:50.339734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee73f5c-8157-40bd-9fd9-57ff5c5f840f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:50.347971 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 390/764] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:56:50 2022] GET /identity/v3/role_assignments?user.id=257450db238d4efda783e30cbcd23d4f&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:50.347971 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c11dc-b49e-4385-b6fb-8b9d3e1d8568 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:50.348245 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c11dc-b49e-4385-b6fb-8b9d3e1d8568 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:50.348573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c11dc-b49e-4385-b6fb-8b9d3e1d8568 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:50.354144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 391/765] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:50.354144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c11dc-b49e-4385-b6fb-8b9d3e1d8568 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:50.366706 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b74988-a2cc-4088-b09e-002dc1a410c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:50.367811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b74988-a2cc-4088-b09e-002dc1a410c0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:50.367811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b74988-a2cc-4088-b09e-002dc1a410c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:50.368045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b74988-a2cc-4088-b09e-002dc1a410c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:50.368794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b74988-a2cc-4088-b09e-002dc1a410c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:50.369138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b74988-a2cc-4088-b09e-002dc1a410c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:50.371568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b74988-a2cc-4088-b09e-002dc1a410c0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:50.397150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 375/766] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:50.398952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3f094-9863-43e5-9d56-cc7c5d167d89 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:50.399126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3f094-9863-43e5-9d56-cc7c5d167d89 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:50.399474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3f094-9863-43e5-9d56-cc7c5d167d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:50.424573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e7d3f094-9863-43e5-9d56-cc7c5d167d89 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:50.426527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e7d3f094-9863-43e5-9d56-cc7c5d167d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:50.450723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:50.469438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:50.524975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c11dc-b49e-4385-b6fb-8b9d3e1d8568 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:50.540017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 376/767] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:56:50.540017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3f094-9863-43e5-9d56-cc7c5d167d89 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:50.583565 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0aa984-72ad-4d0a-8333-cbfa8a376a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:50.585448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0aa984-72ad-4d0a-8333-cbfa8a376a3f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:50.585801 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0aa984-72ad-4d0a-8333-cbfa8a376a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:50.586147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0aa984-72ad-4d0a-8333-cbfa8a376a3f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:50.586932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0aa984-72ad-4d0a-8333-cbfa8a376a3f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:50.588489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0aa984-72ad-4d0a-8333-cbfa8a376a3f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-2a0aa984-72ad-4d0a-8333-cbfa8a376a3f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:50.606464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:50.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:50.608982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:50.608982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:50.608982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:50.608982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:50.608982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:50.608982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:50.608982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:50.608982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:50.608982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:50.608982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:50.608982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:50.608982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:56:50.608982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:56:50.608982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:56:50.608982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:50.612002 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 392/768] 10.222.0.13 () {56 vars in 1129 bytes} [Thu Jul 21 08:56:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:50.613014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0aa984-72ad-4d0a-8333-cbfa8a376a3f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:50.683670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3f094-9863-43e5-9d56-cc7c5d167d89 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:50.740521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115ebe1-6ada-4580-8e11-1f1328662733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:50.741634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115ebe1-6ada-4580-8e11-1f1328662733 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:50.741917 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115ebe1-6ada-4580-8e11-1f1328662733 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:50.742192 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115ebe1-6ada-4580-8e11-1f1328662733 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:50.753750 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115ebe1-6ada-4580-8e11-1f1328662733 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:50.755821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115ebe1-6ada-4580-8e11-1f1328662733 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:50.766098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0aa984-72ad-4d0a-8333-cbfa8a376a3f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:50.777507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-9115ebe1-6ada-4580-8e11-1f1328662733 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:50.777507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:50.777507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:50.777507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:50.777507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:50.777507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:50.777507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:50.777507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:50.777507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:50.777507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:50.777507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:50.777507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:50.777507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:50.777507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:50.777507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:50.778436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:50.779332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:50.779332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:50.779332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:50.779332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:50.779332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:50.779332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:50.779332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:50.779332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:50.780560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 377/769] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:50.782046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115ebe1-6ada-4580-8e11-1f1328662733 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:50.800786 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:50.819181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:50.828243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b1eda-a783-4baa-85e7-49c8f4d149ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:50.829956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b1eda-a783-4baa-85e7-49c8f4d149ff None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:50.830434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b1eda-a783-4baa-85e7-49c8f4d149ff None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:50.830864 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b1eda-a783-4baa-85e7-49c8f4d149ff None admin] PATH_INFO: `/v3/services/network` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:50.831669 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b1eda-a783-4baa-85e7-49c8f4d149ff None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:50.834218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416b1eda-a783-4baa-85e7-49c8f4d149ff None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-416b1eda-a783-4baa-85e7-49c8f4d149ff None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:50.850967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:50.852194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:50.853020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:50.853020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:50.853020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:50.853020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:50.853020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:50.853020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:50.853020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:50.853020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:50.853020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:50.853020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:50.853020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:56:50.853020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:56:50.853020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:56:50.853020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:50.853735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 393/770] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:50 2022] GET /identity/v3/services/network => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:50.854576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b1eda-a783-4baa-85e7-49c8f4d149ff None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:50.933309 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115ebe1-6ada-4580-8e11-1f1328662733 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:50.974012 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5558111-733f-4ed8-b1d6-0464071a3fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:50.975063 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5558111-733f-4ed8-b1d6-0464071a3fac None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:50.975556 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5558111-733f-4ed8-b1d6-0464071a3fac None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:50.976086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5558111-733f-4ed8-b1d6-0464071a3fac None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:50.977115 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5558111-733f-4ed8-b1d6-0464071a3fac None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:50.977688 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5558111-733f-4ed8-b1d6-0464071a3fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:50.979348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5558111-733f-4ed8-b1d6-0464071a3fac None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:50.989429 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416b1eda-a783-4baa-85e7-49c8f4d149ff None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:50.990001 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 378/771] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jul 21 08:56:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:50.991217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5558111-733f-4ed8-b1d6-0464071a3fac None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:51.034683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1eb941-d850-414a-8e3c-f756beda0873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:51.035367 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1eb941-d850-414a-8e3c-f756beda0873 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:51.035629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1eb941-d850-414a-8e3c-f756beda0873 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:51.035904 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1eb941-d850-414a-8e3c-f756beda0873 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:51.036732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1eb941-d850-414a-8e3c-f756beda0873 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:51.037042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1eb941-d850-414a-8e3c-f756beda0873 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:51.039151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1eb941-d850-414a-8e3c-f756beda0873 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:51.060071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 394/772] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:51.060071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1eb941-d850-414a-8e3c-f756beda0873 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:51.090971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:51.134814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:51.143582 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5558111-733f-4ed8-b1d6-0464071a3fac None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:51.196222 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcce817-8436-4f88-ba4d-890124f61ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:51.196890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcce817-8436-4f88-ba4d-890124f61ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:51.197173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcce817-8436-4f88-ba4d-890124f61ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:51.197443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcce817-8436-4f88-ba4d-890124f61ee4 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:51.198147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcce817-8436-4f88-ba4d-890124f61ee4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:51.198456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcce817-8436-4f88-ba4d-890124f61ee4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:51.199834 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcce817-8436-4f88-ba4d-890124f61ee4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:51.205281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1eb941-d850-414a-8e3c-f756beda0873 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:51.208828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 379/773] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jul 21 08:56:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:51.209906 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcce817-8436-4f88-ba4d-890124f61ee4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:51.243282 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e0aac-5ee2-4196-952a-6cc29cd0baae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:51.244352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e0aac-5ee2-4196-952a-6cc29cd0baae None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:51.244352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e0aac-5ee2-4196-952a-6cc29cd0baae None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:51.244556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e0aac-5ee2-4196-952a-6cc29cd0baae None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:51.245268 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e0aac-5ee2-4196-952a-6cc29cd0baae None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:51.245568 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e0aac-5ee2-4196-952a-6cc29cd0baae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:51.246984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e0aac-5ee2-4196-952a-6cc29cd0baae None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:51.258319 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 395/774] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jul 21 08:56:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:51.259252 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e0aac-5ee2-4196-952a-6cc29cd0baae None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:51.347360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcce817-8436-4f88-ba4d-890124f61ee4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:51.404638 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f738dca-d060-473e-8ac9-12fad6023f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:51.406566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f738dca-d060-473e-8ac9-12fad6023f0a None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:51.407041 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f738dca-d060-473e-8ac9-12fad6023f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:51.407508 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f738dca-d060-473e-8ac9-12fad6023f0a None admin] PATH_INFO: `/v3/projects/7fe0f489cbce43859f050c3270e00709/users/c48d41dcf9b344a8b2fec77f4dd75fcb/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:51.410763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f738dca-d060-473e-8ac9-12fad6023f0a None admin] RBAC: Authorizing `identity:create_grant(project_id=7fe0f489cbce43859f050c3270e00709, user_id=c48d41dcf9b344a8b2fec77f4dd75fcb, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:51.413494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f738dca-d060-473e-8ac9-12fad6023f0a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:51.428206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:51.431257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e0aac-5ee2-4196-952a-6cc29cd0baae None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:51.444121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 380/775] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:56:51 2022] PUT /identity/v3/projects/7fe0f489cbce43859f050c3270e00709/users/c48d41dcf9b344a8b2fec77f4dd75fcb/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:56:51.445651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f738dca-d060-473e-8ac9-12fad6023f0a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:51.474305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:51.506795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe25e9-b358-4b25-9dc7-1bffdc7a8b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:51.508309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe25e9-b358-4b25-9dc7-1bffdc7a8b23 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:51.509825 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe25e9-b358-4b25-9dc7-1bffdc7a8b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:51.510736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe25e9-b358-4b25-9dc7-1bffdc7a8b23 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:51.512333 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe25e9-b358-4b25-9dc7-1bffdc7a8b23 None admin] RBAC: Adding query filter params (type=network) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:51.513263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe25e9-b358-4b25-9dc7-1bffdc7a8b23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:51.517481 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe25e9-b358-4b25-9dc7-1bffdc7a8b23 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:51.528332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 396/776] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jul 21 08:56:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:51.543897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe25e9-b358-4b25-9dc7-1bffdc7a8b23 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:51.624955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f738dca-d060-473e-8ac9-12fad6023f0a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:51.660474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add885e-8e52-4a35-a1dc-d86c2956b16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:51.661155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add885e-8e52-4a35-a1dc-d86c2956b16e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:51.661442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add885e-8e52-4a35-a1dc-d86c2956b16e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:51.661721 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add885e-8e52-4a35-a1dc-d86c2956b16e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:51.662726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add885e-8e52-4a35-a1dc-d86c2956b16e None admin] RBAC: Adding query filter params (interface=public, service_id=e14a8a3be5a04924a66fe378159f4c28, region_id=RegionOne) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:51.662879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add885e-8e52-4a35-a1dc-d86c2956b16e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:51.664311 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add885e-8e52-4a35-a1dc-d86c2956b16e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:51.673381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 381/777] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jul 21 08:56:51 2022] GET /identity/v3/endpoints?service_id=e14a8a3be5a04924a66fe378159f4c28&interface=public®ion_id=RegionOne => generated 194 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:51.685576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add885e-8e52-4a35-a1dc-d86c2956b16e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:51.798465 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe25e9-b358-4b25-9dc7-1bffdc7a8b23 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:51.837098 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:51.853292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db032b6-0aee-4ebd-bcc3-4b20ccc32a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:51.854120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db032b6-0aee-4ebd-bcc3-4b20ccc32a0d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:51.854673 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db032b6-0aee-4ebd-bcc3-4b20ccc32a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:51.855052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db032b6-0aee-4ebd-bcc3-4b20ccc32a0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:51.856798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db032b6-0aee-4ebd-bcc3-4b20ccc32a0d None admin] RBAC: Adding query filter params (interface=public, service_id=c4888ae4681444588723a84f7b462a8c, region_id=RegionOne) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:51.857219 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db032b6-0aee-4ebd-bcc3-4b20ccc32a0d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:51.858742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db032b6-0aee-4ebd-bcc3-4b20ccc32a0d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:51.869074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 397/778] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jul 21 08:56:51 2022] GET /identity/v3/endpoints?service_id=c4888ae4681444588723a84f7b462a8c&interface=public®ion_id=RegionOne => generated 194 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:51.872580 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95685452-612d-4003-bc4f-ceb241b43641 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:51.872875 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95685452-612d-4003-bc4f-ceb241b43641 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:51.873179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95685452-612d-4003-bc4f-ceb241b43641 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:51.876947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 398/779] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:51.877868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95685452-612d-4003-bc4f-ceb241b43641 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:51.908762 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add885e-8e52-4a35-a1dc-d86c2956b16e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:51.945699 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:51.959636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaea2d4-3be3-4599-8ff6-cf34507431b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:51.961038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaea2d4-3be3-4599-8ff6-cf34507431b3 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:51.961760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaea2d4-3be3-4599-8ff6-cf34507431b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:51.962184 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaea2d4-3be3-4599-8ff6-cf34507431b3 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:51.963288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaea2d4-3be3-4599-8ff6-cf34507431b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:51.965587 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaea2d4-3be3-4599-8ff6-cf34507431b3 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:51.979208 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 382/780] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jul 21 08:56:51 2022] GET /identity/v3/services => generated 772 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:51.979208 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bc7a2-5cee-428b-85c6-7ccd9ed74700 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:51.979208 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bc7a2-5cee-428b-85c6-7ccd9ed74700 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:51.979208 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bc7a2-5cee-428b-85c6-7ccd9ed74700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:52.016063 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2e3bc7a2-5cee-428b-85c6-7ccd9ed74700 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:52.017457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2e3bc7a2-5cee-428b-85c6-7ccd9ed74700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:52.084347 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95685452-612d-4003-bc4f-ceb241b43641 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:52.141051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 383/781] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:56:52.157880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabbbb9-e26f-460b-821a-9d464e71706f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:52.158562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabbbb9-e26f-460b-821a-9d464e71706f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:52.158849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabbbb9-e26f-460b-821a-9d464e71706f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:52.159125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabbbb9-e26f-460b-821a-9d464e71706f None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:52.159821 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabbbb9-e26f-460b-821a-9d464e71706f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:52.161282 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabbbb9-e26f-460b-821a-9d464e71706f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:52.169520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 399/782] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jul 21 08:56:51 2022] GET /identity/v3/services => generated 772 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:52.171425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bc7a2-5cee-428b-85c6-7ccd9ed74700 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:52.265510 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:52.326326 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bc7a2-5cee-428b-85c6-7ccd9ed74700 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:52.381491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1a29d-bb7b-41a9-b840-5c3dd4e8b259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:52.382322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1a29d-bb7b-41a9-b840-5c3dd4e8b259 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:52.382612 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1a29d-bb7b-41a9-b840-5c3dd4e8b259 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:52.382890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1a29d-bb7b-41a9-b840-5c3dd4e8b259 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:52.401607 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1a29d-bb7b-41a9-b840-5c3dd4e8b259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-96f1a29d-bb7b-41a9-b840-5c3dd4e8b259 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:52.410225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:52.411286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:52.412147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:52.412147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:52.412147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:52.412147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:52.412147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:52.412147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:52.412147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:52.412147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:52.412147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:52.412147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:52.412147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:52.412147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:52.412147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 384/783] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:56:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:52.417911 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabbbb9-e26f-460b-821a-9d464e71706f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:52.568059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabbbb9-e26f-460b-821a-9d464e71706f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:52.586359 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:52.625764 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32240169-21bc-4948-a90e-5efd7bf1d5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:52.626445 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32240169-21bc-4948-a90e-5efd7bf1d5ac None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:52.626760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32240169-21bc-4948-a90e-5efd7bf1d5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:52.627037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32240169-21bc-4948-a90e-5efd7bf1d5ac None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:52.627869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32240169-21bc-4948-a90e-5efd7bf1d5ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:52.628356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32240169-21bc-4948-a90e-5efd7bf1d5ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:52.629935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32240169-21bc-4948-a90e-5efd7bf1d5ac None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:52.644783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 400/784] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:52.650205 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1a29d-bb7b-41a9-b840-5c3dd4e8b259 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:52.729488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b75a93-61c5-439b-bcdb-334990c6d992 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:52.729795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b75a93-61c5-439b-bcdb-334990c6d992 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:52.730071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b75a93-61c5-439b-bcdb-334990c6d992 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:52.736027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 401/785] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:56:52.745849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718c11d-4784-4083-9823-799eb947b6d3 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:52.746156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718c11d-4784-4083-9823-799eb947b6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:52.746422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718c11d-4784-4083-9823-799eb947b6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:52.772506 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1a29d-bb7b-41a9-b840-5c3dd4e8b259 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:52.784111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-5718c11d-4784-4083-9823-799eb947b6d3 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:52.785326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5718c11d-4784-4083-9823-799eb947b6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:52.798482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:52.834531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef2383-ea60-4891-8114-198bb3deffe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:52.835413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef2383-ea60-4891-8114-198bb3deffe8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:52.835686 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef2383-ea60-4891-8114-198bb3deffe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:52.835990 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef2383-ea60-4891-8114-198bb3deffe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:52.862964 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef2383-ea60-4891-8114-198bb3deffe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:52.866653 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef2383-ea60-4891-8114-198bb3deffe8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:52.892992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 402/786] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:56:52.896114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-5bef2383-ea60-4891-8114-198bb3deffe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:52.896114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:52.896114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:52.896114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:52.896114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:52.896114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:52.896114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:52.896114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:52.896114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:52.896114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:52.896114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:52.896114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:52.896114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:52.896114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:52.896114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:52.897011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:52.897882 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:52.897882 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:52.897882 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:52.897882 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:52.897882 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 385/787] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:52.907023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718c11d-4784-4083-9823-799eb947b6d3 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:52.951722 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef2383-ea60-4891-8114-198bb3deffe8 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:53.110496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718c11d-4784-4083-9823-799eb947b6d3 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:53.113662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:53.153611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38c407-8d8a-4d11-89d1-777e2a9f6c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:53.154285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38c407-8d8a-4d11-89d1-777e2a9f6c1f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:53.154580 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38c407-8d8a-4d11-89d1-777e2a9f6c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:53.154848 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38c407-8d8a-4d11-89d1-777e2a9f6c1f None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:53.154995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef2383-ea60-4891-8114-198bb3deffe8 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:53.156039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38c407-8d8a-4d11-89d1-777e2a9f6c1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:53.156361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38c407-8d8a-4d11-89d1-777e2a9f6c1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:53.158077 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38c407-8d8a-4d11-89d1-777e2a9f6c1f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:53.171989 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 403/788] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:53.182406 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38c407-8d8a-4d11-89d1-777e2a9f6c1f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:53.264551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae873d85-1f47-438f-8329-276ea72bd8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:53.266086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae873d85-1f47-438f-8329-276ea72bd8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:53.266371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae873d85-1f47-438f-8329-276ea72bd8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:53.266647 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae873d85-1f47-438f-8329-276ea72bd8d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:53.307198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae873d85-1f47-438f-8329-276ea72bd8d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:53.309121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38c407-8d8a-4d11-89d1-777e2a9f6c1f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-ae873d85-1f47-438f-8329-276ea72bd8d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:53.324980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:53.326031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:53.326878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:53.326878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:53.326878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:53.326878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:53.326878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:53.326878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:53.326878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:53.326878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:53.326878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:53.326878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:53.326878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:53.326878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:53.326878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 386/789] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:53.333723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae873d85-1f47-438f-8329-276ea72bd8d8 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:53.371792 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217fb05-a12c-48df-9b36-5e1bb1d9c1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:53.372859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217fb05-a12c-48df-9b36-5e1bb1d9c1dc None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:53.373201 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217fb05-a12c-48df-9b36-5e1bb1d9c1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:53.373644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8217fb05-a12c-48df-9b36-5e1bb1d9c1dc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:53.396751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217fb05-a12c-48df-9b36-5e1bb1d9c1dc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:53.398968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8217fb05-a12c-48df-9b36-5e1bb1d9c1dc None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:53.412375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-8217fb05-a12c-48df-9b36-5e1bb1d9c1dc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:53.440954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:53.442098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:53.443137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:53.443137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:53.443137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:53.443137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:53.443137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:53.443137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:53.443137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:53.443137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:53.443137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:56:53.443137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:53.443137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 404/790] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:56:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:53.449000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217fb05-a12c-48df-9b36-5e1bb1d9c1dc None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:53.512417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:53.543943 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae873d85-1f47-438f-8329-276ea72bd8d8 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:53.608250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8217fb05-a12c-48df-9b36-5e1bb1d9c1dc None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:53.647727 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e8848-4603-4ab7-acd6-7ba571af3a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:53.648442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e8848-4603-4ab7-acd6-7ba571af3a37 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:53.648726 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e8848-4603-4ab7-acd6-7ba571af3a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:53.648866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f0e7a-15c7-4618-9a13-393e8fc946aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:53.649060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e8848-4603-4ab7-acd6-7ba571af3a37 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:53.649561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f0e7a-15c7-4618-9a13-393e8fc946aa None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:53.649849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e8848-4603-4ab7-acd6-7ba571af3a37 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:53.649951 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f0e7a-15c7-4618-9a13-393e8fc946aa None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:53.650154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e8848-4603-4ab7-acd6-7ba571af3a37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:53.650251 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f0e7a-15c7-4618-9a13-393e8fc946aa None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:53.650960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f0e7a-15c7-4618-9a13-393e8fc946aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:53.651267 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f0e7a-15c7-4618-9a13-393e8fc946aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:53.651593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e8848-4603-4ab7-acd6-7ba571af3a37 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:53.661295 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f0e7a-15c7-4618-9a13-393e8fc946aa None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:53.675218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 405/791] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jul 21 08:56:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:53.677960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 387/792] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:56:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:53.683499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e8848-4603-4ab7-acd6-7ba571af3a37 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:53.684905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f0e7a-15c7-4618-9a13-393e8fc946aa None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:53.813200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:53.825071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f0e7a-15c7-4618-9a13-393e8fc946aa None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:53.867089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fd95c-3596-4dc4-96c1-bc6260d18ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:53.868814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fd95c-3596-4dc4-96c1-bc6260d18ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:53.869444 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fd95c-3596-4dc4-96c1-bc6260d18ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:53.869904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fd95c-3596-4dc4-96c1-bc6260d18ee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:53.882872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fd95c-3596-4dc4-96c1-bc6260d18ee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:53.884876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fd95c-3596-4dc4-96c1-bc6260d18ee6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:53.908200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-4f4fd95c-3596-4dc4-96c1-bc6260d18ee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:53.908200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:53.908200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:53.908200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:53.908200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:53.908200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:53.908200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:53.908200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:53.908200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:53.908200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:53.908200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:53.908200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:53.908200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:53.908200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:53.908200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:53.909110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:53.909971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:53.909971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:53.909971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:53.909971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:53.912212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 388/793] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:53.919359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fd95c-3596-4dc4-96c1-bc6260d18ee6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:53.936293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:53.970883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e8848-4603-4ab7-acd6-7ba571af3a37 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:54.067985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fd95c-3596-4dc4-96c1-bc6260d18ee6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:54.109169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939da6a-cbe9-4581-8f24-d185d63a8fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:54.110341 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939da6a-cbe9-4581-8f24-d185d63a8fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:54.110807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939da6a-cbe9-4581-8f24-d185d63a8fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:54.111284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939da6a-cbe9-4581-8f24-d185d63a8fe8 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:54.112259 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939da6a-cbe9-4581-8f24-d185d63a8fe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:54.112754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939da6a-cbe9-4581-8f24-d185d63a8fe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:54.114422 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939da6a-cbe9-4581-8f24-d185d63a8fe8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:54.116435 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe5712-f8f1-400d-9a8f-49b89e9ad002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:54.124687 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe5712-f8f1-400d-9a8f-49b89e9ad002 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:54.125251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe5712-f8f1-400d-9a8f-49b89e9ad002 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:54.125839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe5712-f8f1-400d-9a8f-49b89e9ad002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:54.131360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 389/794] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:54.138181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939da6a-cbe9-4581-8f24-d185d63a8fe8 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:54.152803 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:54.165431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe5712-f8f1-400d-9a8f-49b89e9ad002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:54.167526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe5712-f8f1-400d-9a8f-49b89e9ad002 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:54.240897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-7afe5712-f8f1-400d-9a8f-49b89e9ad002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:54.240897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:54.240897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:54.240897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:54.240897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:54.240897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:54.240897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:54.240897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:54.240897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:54.240897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:54.240897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:54.240897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:54.240897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:54.240897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:54.240897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:54.241850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:54.242733 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:54.242733 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:54.242733 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:54.242733 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:54.248608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 406/795] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 565 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:54.253081 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe5712-f8f1-400d-9a8f-49b89e9ad002 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:54.283141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939da6a-cbe9-4581-8f24-d185d63a8fe8 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:54.332992 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f5479-3f66-4f49-912d-77e5c0fa3663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:54.334712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f5479-3f66-4f49-912d-77e5c0fa3663 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:54.335253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f5479-3f66-4f49-912d-77e5c0fa3663 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:54.335710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f5479-3f66-4f49-912d-77e5c0fa3663 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:54.346572 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f5479-3f66-4f49-912d-77e5c0fa3663 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:54.349036 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f5479-3f66-4f49-912d-77e5c0fa3663 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-f85f5479-3f66-4f49-912d-77e5c0fa3663 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:54.369219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:54.370206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:54.371210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:54.371210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:54.371210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:54.371210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:54.371210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:54.371210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:54.371210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:54.371210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:54.371210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:56:54.371210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:54.373997 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 390/796] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jul 21 08:56:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:54.380633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2b570-85dd-4dcf-bd23-f9da469f01af None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:54.381163 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2b570-85dd-4dcf-bd23-f9da469f01af None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:54.381606 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2b570-85dd-4dcf-bd23-f9da469f01af None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:54.383564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 391/797] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:54.384645 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2b570-85dd-4dcf-bd23-f9da469f01af None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:54.439885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe5712-f8f1-400d-9a8f-49b89e9ad002 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:54.472759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:54.486146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f05037-50bf-48c2-8507-91a216f8cb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:54.486878 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f05037-50bf-48c2-8507-91a216f8cb9a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:54.487186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f05037-50bf-48c2-8507-91a216f8cb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:54.487476 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f05037-50bf-48c2-8507-91a216f8cb9a None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:54.488241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f05037-50bf-48c2-8507-91a216f8cb9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:54.488544 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f05037-50bf-48c2-8507-91a216f8cb9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:54.490293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f05037-50bf-48c2-8507-91a216f8cb9a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:54.492296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:54.507628 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 407/798] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:54.512070 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9a75c-5997-41ba-8749-5e233fa00d8f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:54.512442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9a75c-5997-41ba-8749-5e233fa00d8f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:54.512874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9a75c-5997-41ba-8749-5e233fa00d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:54.531442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2b570-85dd-4dcf-bd23-f9da469f01af None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:54.536149 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6ee9a75c-5997-41ba-8749-5e233fa00d8f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:54.537986 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6ee9a75c-5997-41ba-8749-5e233fa00d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:54.588825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1ad71-90f7-4c75-a7ce-e747d243bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:54.589518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1ad71-90f7-4c75-a7ce-e747d243bafe None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:54.589929 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1ad71-90f7-4c75-a7ce-e747d243bafe None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:54.590066 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1ad71-90f7-4c75-a7ce-e747d243bafe None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:54.590913 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1ad71-90f7-4c75-a7ce-e747d243bafe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:54.591225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1ad71-90f7-4c75-a7ce-e747d243bafe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:54.593648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1ad71-90f7-4c75-a7ce-e747d243bafe None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:54.624960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 392/799] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jul 21 08:56:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:54.626718 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe3036-a157-49ff-9748-bc6e09729312 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:54.627011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe3036-a157-49ff-9748-bc6e09729312 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:54.627290 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe3036-a157-49ff-9748-bc6e09729312 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:54.631976 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 393/800] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:54.631976 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe3036-a157-49ff-9748-bc6e09729312 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:54.646929 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 408/801] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:56:54.648525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e73374-4657-42e6-a978-f3e54f405039 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:54.648989 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e73374-4657-42e6-a978-f3e54f405039 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:54.649424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e73374-4657-42e6-a978-f3e54f405039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:54.668325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-28e73374-4657-42e6-a978-f3e54f405039 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:54.669884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-28e73374-4657-42e6-a978-f3e54f405039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:54.764165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 409/802] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:56:54.764165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e73374-4657-42e6-a978-f3e54f405039 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:54.810261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe3036-a157-49ff-9748-bc6e09729312 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:54.833131 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:54.839845 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:54.853122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25f0ab-f59e-4a85-93f1-a4c6680aedc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:54.854034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25f0ab-f59e-4a85-93f1-a4c6680aedc4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:54.854320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25f0ab-f59e-4a85-93f1-a4c6680aedc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:54.854593 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25f0ab-f59e-4a85-93f1-a4c6680aedc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:54.863371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25f0ab-f59e-4a85-93f1-a4c6680aedc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:54.864886 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25f0ab-f59e-4a85-93f1-a4c6680aedc4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:54.881460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-ce25f0ab-f59e-4a85-93f1-a4c6680aedc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:54.881460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:54.881460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:54.881460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:54.881460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:54.881460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:54.881460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:54.881460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:54.881460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:54.881460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:54.881460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:54.881460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:54.881460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:54.881460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:54.881460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:54.888573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:54.889477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:54.889477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:54.889477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:54.889477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:54.889477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:54.889477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:54.889477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:54.889477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:54.889477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 394/803] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:54.889477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25f0ab-f59e-4a85-93f1-a4c6680aedc4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:54.903592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e73374-4657-42e6-a978-f3e54f405039 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:54.951896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd3bed-a273-46af-8342-36c2661a4e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:54.953322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd3bed-a273-46af-8342-36c2661a4e75 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:54.953657 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd3bed-a273-46af-8342-36c2661a4e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:54.956545 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd3bed-a273-46af-8342-36c2661a4e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:54.968482 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd3bed-a273-46af-8342-36c2661a4e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:54.970309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd3bed-a273-46af-8342-36c2661a4e75 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:54.995144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-15bd3bed-a273-46af-8342-36c2661a4e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:54.995144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:54.995144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:54.995144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:54.995144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:54.995144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:54.995144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:54.995144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:54.995144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:54.995144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:54.995144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:54.995144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:54.995144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:54.995144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:54.995144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:54.996557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:54.997610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:54.997610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:54.997610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:54.997610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:55.000335 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 410/804] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:55.002297 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd3bed-a273-46af-8342-36c2661a4e75 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:55.033895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25f0ab-f59e-4a85-93f1-a4c6680aedc4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:55.077758 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80151b77-9991-4100-b71e-27efb343b36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:55.079172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80151b77-9991-4100-b71e-27efb343b36b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:55.079461 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80151b77-9991-4100-b71e-27efb343b36b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:55.079736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80151b77-9991-4100-b71e-27efb343b36b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:55.091503 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80151b77-9991-4100-b71e-27efb343b36b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-80151b77-9991-4100-b71e-27efb343b36b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:55.100912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:55.109193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:55.110035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:55.110035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:55.110035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:55.110035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:55.110035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:55.110035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:55.110035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:55.110035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:55.110035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:55.110035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:55.110035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:55.110035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:55.110035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 395/805] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:56:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:55.110035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80151b77-9991-4100-b71e-27efb343b36b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:55.144873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd3bed-a273-46af-8342-36c2661a4e75 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:55.155148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:55.165447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:55.191857 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39807721-09f4-46c5-842a-3b60a1157b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:55.192876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39807721-09f4-46c5-842a-3b60a1157b68 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:55.193269 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39807721-09f4-46c5-842a-3b60a1157b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:55.193654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39807721-09f4-46c5-842a-3b60a1157b68 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:55.194423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39807721-09f4-46c5-842a-3b60a1157b68 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:55.195894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39807721-09f4-46c5-842a-3b60a1157b68 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-39807721-09f4-46c5-842a-3b60a1157b68 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:55.212298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:55.213542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:55.214674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 411/806] 10.222.0.13 () {56 vars in 1129 bytes} [Thu Jul 21 08:56:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:55.215706 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39807721-09f4-46c5-842a-3b60a1157b68 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:55.247517 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80151b77-9991-4100-b71e-27efb343b36b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:55.288057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85c936-5f32-4dd4-bd71-74f532849963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:55.288783 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85c936-5f32-4dd4-bd71-74f532849963 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:55.289094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85c936-5f32-4dd4-bd71-74f532849963 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:55.289368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85c936-5f32-4dd4-bd71-74f532849963 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:55.290208 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85c936-5f32-4dd4-bd71-74f532849963 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:55.290555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85c936-5f32-4dd4-bd71-74f532849963 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:55.292187 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85c936-5f32-4dd4-bd71-74f532849963 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:55.306371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 396/807] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:55.307228 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85c936-5f32-4dd4-bd71-74f532849963 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:55.330092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39807721-09f4-46c5-842a-3b60a1157b68 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:55.372558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ae7e9-b127-419a-89c0-c99a92ef6387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:55.373225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ae7e9-b127-419a-89c0-c99a92ef6387 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:55.373509 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ae7e9-b127-419a-89c0-c99a92ef6387 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:55.373786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ae7e9-b127-419a-89c0-c99a92ef6387 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:55.375478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ae7e9-b127-419a-89c0-c99a92ef6387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:55.375869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ae7e9-b127-419a-89c0-c99a92ef6387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:55.377302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ae7e9-b127-419a-89c0-c99a92ef6387 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:55.390539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 412/808] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:55.391895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b660c-ea07-4266-b8e3-1f5a0db2e798 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:55.392341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b660c-ea07-4266-b8e3-1f5a0db2e798 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:55.392604 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b660c-ea07-4266-b8e3-1f5a0db2e798 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:55.394202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 413/809] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:55.394840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b660c-ea07-4266-b8e3-1f5a0db2e798 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:55.426788 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:55.429903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85c936-5f32-4dd4-bd71-74f532849963 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:55.434457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:55.474971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788be53-526a-4383-8263-8765637123ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:55.475808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788be53-526a-4383-8263-8765637123ab None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:55.476148 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788be53-526a-4383-8263-8765637123ab None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:55.476433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788be53-526a-4383-8263-8765637123ab None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:55.477352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788be53-526a-4383-8263-8765637123ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:55.477671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788be53-526a-4383-8263-8765637123ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:55.479177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788be53-526a-4383-8263-8765637123ab None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:55.489807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 397/810] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:55.490653 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788be53-526a-4383-8263-8765637123ab None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:55.515682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b660c-ea07-4266-b8e3-1f5a0db2e798 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:55.555291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74471777-c01d-415b-b8ca-1a2cf66832a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:55.556064 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74471777-c01d-415b-b8ca-1a2cf66832a2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:55.556405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74471777-c01d-415b-b8ca-1a2cf66832a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:55.556700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74471777-c01d-415b-b8ca-1a2cf66832a2 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:55.557415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74471777-c01d-415b-b8ca-1a2cf66832a2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:55.557720 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74471777-c01d-415b-b8ca-1a2cf66832a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:55.559109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74471777-c01d-415b-b8ca-1a2cf66832a2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:55.564990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 414/811] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jul 21 08:56:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:55.565820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74471777-c01d-415b-b8ca-1a2cf66832a2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:55.642302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788be53-526a-4383-8263-8765637123ab None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:55.675017 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:55.682499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0e4a0-c398-4a94-add4-cd92023ee946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:55.683469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0e4a0-c398-4a94-add4-cd92023ee946 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:55.683801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0e4a0-c398-4a94-add4-cd92023ee946 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:55.684137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0e4a0-c398-4a94-add4-cd92023ee946 None admin] PATH_INFO: `/v3/projects/7fe0f489cbce43859f050c3270e00709/users/061def74ec7f42ce9a174e0944bf499f/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:55.684711 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74471777-c01d-415b-b8ca-1a2cf66832a2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:55.686430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0e4a0-c398-4a94-add4-cd92023ee946 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fe0f489cbce43859f050c3270e00709, user_id=061def74ec7f42ce9a174e0944bf499f, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:55.688058 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0e4a0-c398-4a94-add4-cd92023ee946 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:55.704150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 398/812] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:56:55 2022] PUT /identity/v3/projects/7fe0f489cbce43859f050c3270e00709/users/061def74ec7f42ce9a174e0944bf499f/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:56:55.705826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c2ba2-a2d3-4714-bd7d-183b3597268b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:55.706573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c2ba2-a2d3-4714-bd7d-183b3597268b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:55.706573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c2ba2-a2d3-4714-bd7d-183b3597268b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:55.739477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309bff0-bb35-4503-ada9-b36905be4c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:55.740640 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309bff0-bb35-4503-ada9-b36905be4c8c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:55.741158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309bff0-bb35-4503-ada9-b36905be4c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:55.741502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309bff0-bb35-4503-ada9-b36905be4c8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:55.745698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-822c2ba2-a2d3-4714-bd7d-183b3597268b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:55.747379 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-822c2ba2-a2d3-4714-bd7d-183b3597268b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:55.753600 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309bff0-bb35-4503-ada9-b36905be4c8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:55.755705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309bff0-bb35-4503-ada9-b36905be4c8c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:55.774469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-e309bff0-bb35-4503-ada9-b36905be4c8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:55.774469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:55.774469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:55.774469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:55.774469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:55.774469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:55.774469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:55.774469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:55.774469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:55.774469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:55.774469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:55.774469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:55.774469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:55.774469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:55.774469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:55.775501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:55.776622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:55.776622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:55.776622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:55.776622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:55.776622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:55.776622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:55.776622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:55.776622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:55.782713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 415/813] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:55.783704 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309bff0-bb35-4503-ada9-b36905be4c8c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:55.785663 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:55.852825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 399/814] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:56:55.853747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c2ba2-a2d3-4714-bd7d-183b3597268b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:55.931172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309bff0-bb35-4503-ada9-b36905be4c8c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:55.972879 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab85cad-9b42-48b6-98d1-b997d8c8f7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:55.973759 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab85cad-9b42-48b6-98d1-b997d8c8f7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:55.974047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab85cad-9b42-48b6-98d1-b997d8c8f7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:55.974322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab85cad-9b42-48b6-98d1-b997d8c8f7b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:55.987851 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab85cad-9b42-48b6-98d1-b997d8c8f7b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:55.989399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab85cad-9b42-48b6-98d1-b997d8c8f7b0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:56.009631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-7ab85cad-9b42-48b6-98d1-b997d8c8f7b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:56.009631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:56.009631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:56.009631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:56.009631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:56.009631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:56.009631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:56.009631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:56.009631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:56.009631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:56.009631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:56.009631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:56.009631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:56.009631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:56.009631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:56.010752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:56.011743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:56.011743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:56.011743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:56.011743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:56.012034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 416/815] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:56.018005 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab85cad-9b42-48b6-98d1-b997d8c8f7b0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:56.018188 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c2ba2-a2d3-4714-bd7d-183b3597268b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:56.025482 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:56.063468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ef95a-733d-4a37-b25d-17905d13ee81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:56.064269 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ef95a-733d-4a37-b25d-17905d13ee81 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:56.064561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ef95a-733d-4a37-b25d-17905d13ee81 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:56.064834 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ef95a-733d-4a37-b25d-17905d13ee81 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:56.065585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ef95a-733d-4a37-b25d-17905d13ee81 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:56.065928 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ef95a-733d-4a37-b25d-17905d13ee81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:56.067569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ef95a-733d-4a37-b25d-17905d13ee81 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:56.074528 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 400/816] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jul 21 08:56:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:56.075555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ef95a-733d-4a37-b25d-17905d13ee81 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:56.098977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:56.144230 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab85cad-9b42-48b6-98d1-b997d8c8f7b0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:56.197180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2fcdb-c9dc-4dca-8484-ae45f8657ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:56.198196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2fcdb-c9dc-4dca-8484-ae45f8657ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:56.198342 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2fcdb-c9dc-4dca-8484-ae45f8657ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:56.198683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2fcdb-c9dc-4dca-8484-ae45f8657ff9 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:56.199568 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2fcdb-c9dc-4dca-8484-ae45f8657ff9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:56.199904 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2fcdb-c9dc-4dca-8484-ae45f8657ff9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:56.201598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2fcdb-c9dc-4dca-8484-ae45f8657ff9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:56.230958 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 417/817] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:56.230958 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2fcdb-c9dc-4dca-8484-ae45f8657ff9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:56.244284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ef95a-733d-4a37-b25d-17905d13ee81 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:56.297394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d64464-35c1-4c46-a585-a6f487303979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:56.298178 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d64464-35c1-4c46-a585-a6f487303979 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:56.298534 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d64464-35c1-4c46-a585-a6f487303979 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:56.298843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d64464-35c1-4c46-a585-a6f487303979 None admin] PATH_INFO: `/v3/services/network` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:56.299527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d64464-35c1-4c46-a585-a6f487303979 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:56.301166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d64464-35c1-4c46-a585-a6f487303979 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-c3d64464-35c1-4c46-a585-a6f487303979 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:56.315518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:56.317363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:56.318523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:56.318523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:56.318523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:56.318523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:56.318523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:56.318523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:56.318523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:56.318523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:56.318523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:56.318523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:56.318523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:56:56.318523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:56:56.318523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:56:56.318523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:56.318523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 401/818] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:56 2022] GET /identity/v3/services/network => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:56.319889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d64464-35c1-4c46-a585-a6f487303979 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:56.337250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:56.382893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2fcdb-c9dc-4dca-8484-ae45f8657ff9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:56.424027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:56.425599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80df801-9131-4d9f-9186-65e82bb052c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:56.426408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80df801-9131-4d9f-9186-65e82bb052c0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:56.426664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80df801-9131-4d9f-9186-65e82bb052c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:56.426950 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80df801-9131-4d9f-9186-65e82bb052c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:56.427745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80df801-9131-4d9f-9186-65e82bb052c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:56.428106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80df801-9131-4d9f-9186-65e82bb052c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:56.429539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80df801-9131-4d9f-9186-65e82bb052c0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:56.446814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 418/819] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:56.446814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80df801-9131-4d9f-9186-65e82bb052c0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:56.453919 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d64464-35c1-4c46-a585-a6f487303979 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:56.493312 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceca51-3318-40b2-8c2a-7f7f6d862b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:56.494110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceca51-3318-40b2-8c2a-7f7f6d862b8c None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:56.494468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceca51-3318-40b2-8c2a-7f7f6d862b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:56.494825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceca51-3318-40b2-8c2a-7f7f6d862b8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:56.495670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dceca51-3318-40b2-8c2a-7f7f6d862b8c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:56.497234 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dceca51-3318-40b2-8c2a-7f7f6d862b8c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:56.527956 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 402/820] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jul 21 08:56:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:56:56.527956 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceca51-3318-40b2-8c2a-7f7f6d862b8c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:56.578557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80df801-9131-4d9f-9186-65e82bb052c0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:56.645927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:56.677080 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceca51-3318-40b2-8c2a-7f7f6d862b8c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:56.695606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea5119-eb66-4d0e-bd14-246be6fa8ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:56.697728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea5119-eb66-4d0e-bd14-246be6fa8ddf None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:56.698321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea5119-eb66-4d0e-bd14-246be6fa8ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:56.699244 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea5119-eb66-4d0e-bd14-246be6fa8ddf None admin] PATH_INFO: `/v3/projects/7fe0f489cbce43859f050c3270e00709/users/257450db238d4efda783e30cbcd23d4f/roles/8511cf3e5d2c4acdae6d0ffda5e72c63` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:56.723787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:56.732553 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b698edc-6855-4435-8947-340ede5f48cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:56.734238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b698edc-6855-4435-8947-340ede5f48cc None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:56.734755 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b698edc-6855-4435-8947-340ede5f48cc None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:56.735402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b698edc-6855-4435-8947-340ede5f48cc None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:56.736179 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b698edc-6855-4435-8947-340ede5f48cc None admin] RBAC: Adding query filter params (name=network) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:56.737105 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea5119-eb66-4d0e-bd14-246be6fa8ddf None admin] RBAC: Authorizing `identity:create_grant(project_id=7fe0f489cbce43859f050c3270e00709, user_id=257450db238d4efda783e30cbcd23d4f, role_id=8511cf3e5d2c4acdae6d0ffda5e72c63)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:56.737225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b698edc-6855-4435-8947-340ede5f48cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:56.737918 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b698edc-6855-4435-8947-340ede5f48cc None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:56.739357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea5119-eb66-4d0e-bd14-246be6fa8ddf None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:56.745639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 403/821] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jul 21 08:56:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:56.746523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b698edc-6855-4435-8947-340ede5f48cc None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:56.770748 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 419/822] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:56:56 2022] PUT /identity/v3/projects/7fe0f489cbce43859f050c3270e00709/users/257450db238d4efda783e30cbcd23d4f/roles/8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:56:56.772598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea5119-eb66-4d0e-bd14-246be6fa8ddf None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:56.946041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea5119-eb66-4d0e-bd14-246be6fa8ddf None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:56.992713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc469720-53b9-409b-a23a-3eb113864ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:56.993395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc469720-53b9-409b-a23a-3eb113864ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:56.993682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc469720-53b9-409b-a23a-3eb113864ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:56.993959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc469720-53b9-409b-a23a-3eb113864ae4 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:56.994695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc469720-53b9-409b-a23a-3eb113864ae4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:56.995013 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc469720-53b9-409b-a23a-3eb113864ae4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:56.996467 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc469720-53b9-409b-a23a-3eb113864ae4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:57.002513 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b698edc-6855-4435-8947-340ede5f48cc None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:57.007793 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 420/823] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jul 21 08:56:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:57.014528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc469720-53b9-409b-a23a-3eb113864ae4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:57.052515 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9012a5-6166-48eb-8b10-0720edaebbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:57.053327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9012a5-6166-48eb-8b10-0720edaebbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:57.053621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9012a5-6166-48eb-8b10-0720edaebbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:57.053905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9012a5-6166-48eb-8b10-0720edaebbb2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:57.060724 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:57.063334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9012a5-6166-48eb-8b10-0720edaebbb2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:57.064982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9012a5-6166-48eb-8b10-0720edaebbb2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-6a9012a5-6166-48eb-8b10-0720edaebbb2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:57.082468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:57.083501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:57.084526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:57.084526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:57.084526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:57.084526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:57.084526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:57.084526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:57.084526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:57.084526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:57.084526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:56:57.084526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:57.084526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 404/824] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jul 21 08:56:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:57.091713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9012a5-6166-48eb-8b10-0720edaebbb2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:57.184868 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:57.224035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9012a5-6166-48eb-8b10-0720edaebbb2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:57.285536 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793102e3-db72-4ac3-b957-201952dfa195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:57.286236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793102e3-db72-4ac3-b957-201952dfa195 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:57.286520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793102e3-db72-4ac3-b957-201952dfa195 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:57.286801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793102e3-db72-4ac3-b957-201952dfa195 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:57.287669 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793102e3-db72-4ac3-b957-201952dfa195 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:57.288008 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793102e3-db72-4ac3-b957-201952dfa195 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:57.289549 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793102e3-db72-4ac3-b957-201952dfa195 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:57.320939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc469720-53b9-409b-a23a-3eb113864ae4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:57.326507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 405/825] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jul 21 08:56:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:57.332846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793102e3-db72-4ac3-b957-201952dfa195 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:57.393598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6dc68-99bc-4772-93b5-9f107dc09c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:57.394331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6dc68-99bc-4772-93b5-9f107dc09c3f None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:57.394630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6dc68-99bc-4772-93b5-9f107dc09c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:57.394902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6dc68-99bc-4772-93b5-9f107dc09c3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:57.395760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6dc68-99bc-4772-93b5-9f107dc09c3f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:57.397192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6dc68-99bc-4772-93b5-9f107dc09c3f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:57.452874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 421/826] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jul 21 08:56:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:56:57.459981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793102e3-db72-4ac3-b957-201952dfa195 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:57.484617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:57.509992 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb79d3-49cf-4b8d-871c-c30235749488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:57.510821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb79d3-49cf-4b8d-871c-c30235749488 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:57.511212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb79d3-49cf-4b8d-871c-c30235749488 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:57.511594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb79d3-49cf-4b8d-871c-c30235749488 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:57.599532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb79d3-49cf-4b8d-871c-c30235749488 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:57.601258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb79d3-49cf-4b8d-871c-c30235749488 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:57.730939 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 406/827] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:57 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:56:57.739118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6dc68-99bc-4772-93b5-9f107dc09c3f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:57.903148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:57.919428 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6dc68-99bc-4772-93b5-9f107dc09c3f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:57.993582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071da4e-3f6a-4237-b796-b434656970d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:57.994634 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071da4e-3f6a-4237-b796-b434656970d9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:57.995031 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071da4e-3f6a-4237-b796-b434656970d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:57.995422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071da4e-3f6a-4237-b796-b434656970d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:58.013890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071da4e-3f6a-4237-b796-b434656970d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:58.016787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071da4e-3f6a-4237-b796-b434656970d9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:58.049873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-e071da4e-3f6a-4237-b796-b434656970d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:58.049873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:58.049873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:58.049873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:58.049873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:58.049873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:58.049873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:58.049873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:58.049873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:58.049873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:58.049873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:58.049873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:58.049873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:58.049873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:58.049873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:58.050834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:58.051708 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:58.051708 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:58.051708 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:58.051708 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:58.052761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 422/828] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:58.058083 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb79d3-49cf-4b8d-871c-c30235749488 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:58.069995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fea01c-9358-42bf-b24a-65d1e72b973d None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:58.071435 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fea01c-9358-42bf-b24a-65d1e72b973d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:58.072678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fea01c-9358-42bf-b24a-65d1e72b973d None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:58.076768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:58.078511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 423/829] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:58 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:56:58.087453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91021d-c760-4a05-8cb6-f8d1d3d52fc4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:58.088316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91021d-c760-4a05-8cb6-f8d1d3d52fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:58.089045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91021d-c760-4a05-8cb6-f8d1d3d52fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:58.116650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-df91021d-c760-4a05-8cb6-f8d1d3d52fc4 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:58.118982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-df91021d-c760-4a05-8cb6-f8d1d3d52fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:58.189118 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb79d3-49cf-4b8d-871c-c30235749488 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:58.200019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 424/830] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:56:58.221114 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91021d-c760-4a05-8cb6-f8d1d3d52fc4 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:58.230744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb91280-141a-4f44-9ac2-908c651f45fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:58.231427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb91280-141a-4f44-9ac2-908c651f45fc None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:58.231715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb91280-141a-4f44-9ac2-908c651f45fc None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:58.232024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb91280-141a-4f44-9ac2-908c651f45fc None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:58.232757 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb91280-141a-4f44-9ac2-908c651f45fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:58.233064 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb91280-141a-4f44-9ac2-908c651f45fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:58.235287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb91280-141a-4f44-9ac2-908c651f45fc None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:58.251064 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 407/831] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:58.256512 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb91280-141a-4f44-9ac2-908c651f45fc None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:58.310752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:58.339088 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91021d-c760-4a05-8cb6-f8d1d3d52fc4 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:58.372493 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:58.393877 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2305faa-caf7-452e-944f-c419d0b9f55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:58.394992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2305faa-caf7-452e-944f-c419d0b9f55b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:58.395366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2305faa-caf7-452e-944f-c419d0b9f55b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:58.395692 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2305faa-caf7-452e-944f-c419d0b9f55b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:58.414507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2305faa-caf7-452e-944f-c419d0b9f55b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:58.421890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb91280-141a-4f44-9ac2-908c651f45fc None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-e2305faa-caf7-452e-944f-c419d0b9f55b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:58.429357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:58.430511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:58.431466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:58.431466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:58.431466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:58.431466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:58.431466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:58.431466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:58.431466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:58.431466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:58.431466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:58.431466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:58.431466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:56:58.431466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:58.432259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 425/832] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:56:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:58.441296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2305faa-caf7-452e-944f-c419d0b9f55b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:58.478464 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cdc97-d971-4783-ac61-57b96999e051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:58.479412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cdc97-d971-4783-ac61-57b96999e051 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:58.479717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cdc97-d971-4783-ac61-57b96999e051 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:58.480068 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cdc97-d971-4783-ac61-57b96999e051 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:58.490812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cdc97-d971-4783-ac61-57b96999e051 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:58.492681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cdc97-d971-4783-ac61-57b96999e051 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:58.513682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-477cdc97-d971-4783-ac61-57b96999e051 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:58.513682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:58.513682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:58.513682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:58.513682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:58.513682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:58.513682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:58.513682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:58.513682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:58.513682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:58.513682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:58.513682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:58.513682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:56:58.513682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:56:58.513682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:58.516482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:58.518592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:56:58.518592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:56:58.518592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:56:58.518592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:56:58.518592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:56:58.518592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:56:58.518592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:56:58.518592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:58.521188 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 408/833] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:56:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:58.522754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cdc97-d971-4783-ac61-57b96999e051 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:58.611242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2305faa-caf7-452e-944f-c419d0b9f55b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:58.661488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6d267-baad-4bd3-b451-ff9da873405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:58.662799 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6d267-baad-4bd3-b451-ff9da873405f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:58.663089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6d267-baad-4bd3-b451-ff9da873405f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:58.663395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6d267-baad-4bd3-b451-ff9da873405f None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:58.664265 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6d267-baad-4bd3-b451-ff9da873405f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:58.664574 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6d267-baad-4bd3-b451-ff9da873405f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:58.665610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cdc97-d971-4783-ac61-57b96999e051 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:58.665955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6d267-baad-4bd3-b451-ff9da873405f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:58.667715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:58.684019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 426/834] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:56:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:58.686272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6d267-baad-4bd3-b451-ff9da873405f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:58.708902 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:58.724815 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906f97e-0bf6-4140-830d-61617ad42f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:58.725950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906f97e-0bf6-4140-830d-61617ad42f56 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:58.726433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906f97e-0bf6-4140-830d-61617ad42f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:58.726964 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906f97e-0bf6-4140-830d-61617ad42f56 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:58.728018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906f97e-0bf6-4140-830d-61617ad42f56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:58.728613 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906f97e-0bf6-4140-830d-61617ad42f56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:58.730391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906f97e-0bf6-4140-830d-61617ad42f56 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:58.750027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 409/835] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:56:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:58.751387 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906f97e-0bf6-4140-830d-61617ad42f56 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:58.803677 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6d267-baad-4bd3-b451-ff9da873405f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:58.841625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d5bdc-7f97-493c-865a-0afb0d1438e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:58.842829 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d5bdc-7f97-493c-865a-0afb0d1438e1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:58.842829 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d5bdc-7f97-493c-865a-0afb0d1438e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:58.843159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d5bdc-7f97-493c-865a-0afb0d1438e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:58.855609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d5bdc-7f97-493c-865a-0afb0d1438e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:58.857122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d5bdc-7f97-493c-865a-0afb0d1438e1 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:58.870323 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906f97e-0bf6-4140-830d-61617ad42f56 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:58.877718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-a74d5bdc-7f97-493c-865a-0afb0d1438e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:58.877718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:58.877718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:58.877718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:58.877718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:58.877718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:58.877718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:58.877718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:58.877718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:58.877718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:58.877718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:58.877718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:58.877718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:56:58.877718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:56:58.877718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:58.878622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:58.879491 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:56:58.879491 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:56:58.879491 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:56:58.879491 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:58.884092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 427/836] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:56:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:58.885970 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d5bdc-7f97-493c-865a-0afb0d1438e1 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:58.912643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d240a7-1670-4394-921b-3ed47ac3d900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:58.913657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d240a7-1670-4394-921b-3ed47ac3d900 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:58.914112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d240a7-1670-4394-921b-3ed47ac3d900 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:58.914557 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d240a7-1670-4394-921b-3ed47ac3d900 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:58.915671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d240a7-1670-4394-921b-3ed47ac3d900 None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=c48d41dcf9b344a8b2fec77f4dd75fcb) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:58.916224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d240a7-1670-4394-921b-3ed47ac3d900 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:58.917977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d240a7-1670-4394-921b-3ed47ac3d900 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:58.932098 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 410/837] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:56:58 2022] GET /identity/v3/role_assignments?user.id=c48d41dcf9b344a8b2fec77f4dd75fcb&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=000e942517f2469ba36d2e65978bc314 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:58.936575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb933a23-281d-4e47-8b92-686161fd74bf None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:58.937541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb933a23-281d-4e47-8b92-686161fd74bf None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:58.937785 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb933a23-281d-4e47-8b92-686161fd74bf None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:58.944344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 411/838] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:58 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:56:58.946067 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9bd9b-55e3-4ea1-bc06-4363017eb4ed None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:58.946529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9bd9b-55e3-4ea1-bc06-4363017eb4ed None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:58.946992 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9bd9b-55e3-4ea1-bc06-4363017eb4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:58.956397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:58.983358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f4f9bd9b-55e3-4ea1-bc06-4363017eb4ed None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:58.986728 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f4f9bd9b-55e3-4ea1-bc06-4363017eb4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:59.013717 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d5bdc-7f97-493c-865a-0afb0d1438e1 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:59.061088 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31ca77-7a96-4abe-a8f9-d52d954d980d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:59.062350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31ca77-7a96-4abe-a8f9-d52d954d980d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:59.062863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31ca77-7a96-4abe-a8f9-d52d954d980d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:59.063298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31ca77-7a96-4abe-a8f9-d52d954d980d None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:59.064437 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31ca77-7a96-4abe-a8f9-d52d954d980d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:59.064824 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31ca77-7a96-4abe-a8f9-d52d954d980d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:59.066130 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:59.068413 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31ca77-7a96-4abe-a8f9-d52d954d980d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:59.092033 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 428/839] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:59.095356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31ca77-7a96-4abe-a8f9-d52d954d980d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:59.104410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 412/840] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:56:59.125395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9bd9b-55e3-4ea1-bc06-4363017eb4ed None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:59.239971 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31ca77-7a96-4abe-a8f9-d52d954d980d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:59.253180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:59.287020 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9bd9b-55e3-4ea1-bc06-4363017eb4ed None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:59.302281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcf54c-2568-44ed-9c80-8ed4ca8dd657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:59.303364 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcf54c-2568-44ed-9c80-8ed4ca8dd657 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:59.303832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcf54c-2568-44ed-9c80-8ed4ca8dd657 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:59.304215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcf54c-2568-44ed-9c80-8ed4ca8dd657 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:59.314396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcf54c-2568-44ed-9c80-8ed4ca8dd657 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:59.316424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bcf54c-2568-44ed-9c80-8ed4ca8dd657 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-44bcf54c-2568-44ed-9c80-8ed4ca8dd657 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:56:59.334613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:59.335685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:59.336859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:59.336859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:59.336859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:56:59.336859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:56:59.336859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:56:59.336859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:56:59.336859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:56:59.336859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:56:59.336859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:56:59.336859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:56:59.341456 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 429/841] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:56:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:59.343378 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcf54c-2568-44ed-9c80-8ed4ca8dd657 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:59.478236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50701af-f29c-46bf-99b9-eb578e857995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:59.479365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50701af-f29c-46bf-99b9-eb578e857995 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:59.479850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50701af-f29c-46bf-99b9-eb578e857995 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:59.480183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50701af-f29c-46bf-99b9-eb578e857995 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:59.480956 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50701af-f29c-46bf-99b9-eb578e857995 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:59.482553 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50701af-f29c-46bf-99b9-eb578e857995 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:59.482714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bcf54c-2568-44ed-9c80-8ed4ca8dd657 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-c50701af-f29c-46bf-99b9-eb578e857995 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:59.499454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:59.501061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:59.502507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:59.502507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:59.502507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:59.502507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:59.502507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:59.502507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:59.502507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:59.502507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:59.502507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:59.502507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:59.502507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:56:59.502507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:56:59.502507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:56:59.502507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:59.505175 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 413/842] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:56:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:56:59.507524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5147ce7-72ad-43c9-88a7-b45825fb191b None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:59.507992 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5147ce7-72ad-43c9-88a7-b45825fb191b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:59.508333 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5147ce7-72ad-43c9-88a7-b45825fb191b None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:59.512135 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 414/843] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:56:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:56:59.512135 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5147ce7-72ad-43c9-88a7-b45825fb191b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:59.537695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086459e6-9000-4d49-a432-90381d7c66ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:59.538445 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086459e6-9000-4d49-a432-90381d7c66ff None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:59.538800 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086459e6-9000-4d49-a432-90381d7c66ff None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:59.539084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086459e6-9000-4d49-a432-90381d7c66ff None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:59.540160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086459e6-9000-4d49-a432-90381d7c66ff None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:59.540510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086459e6-9000-4d49-a432-90381d7c66ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:59.542141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086459e6-9000-4d49-a432-90381d7c66ff None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:59.564969 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 430/844] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jul 21 08:56:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:56:59.565955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115901f-b46c-4aeb-ac2f-d7aa0e90228e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:59.566258 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115901f-b46c-4aeb-ac2f-d7aa0e90228e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:59.566534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115901f-b46c-4aeb-ac2f-d7aa0e90228e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:59.584092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:59.593795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e115901f-b46c-4aeb-ac2f-d7aa0e90228e None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:56:59.595532 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e115901f-b46c-4aeb-ac2f-d7aa0e90228e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:56:59.624925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:59.661159 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5147ce7-72ad-43c9-88a7-b45825fb191b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:59.683992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 431/845] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:56:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:56:59.684274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115901f-b46c-4aeb-ac2f-d7aa0e90228e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:59.731965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839abc6-e347-4274-b34b-3df0cbf0bd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:59.733129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839abc6-e347-4274-b34b-3df0cbf0bd0f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:59.734128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839abc6-e347-4274-b34b-3df0cbf0bd0f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:59.735175 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839abc6-e347-4274-b34b-3df0cbf0bd0f None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:59.735594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839abc6-e347-4274-b34b-3df0cbf0bd0f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:56:59.735973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839abc6-e347-4274-b34b-3df0cbf0bd0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:59.738529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839abc6-e347-4274-b34b-3df0cbf0bd0f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:59.746456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 415/846] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:56:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:56:59.750739 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839abc6-e347-4274-b34b-3df0cbf0bd0f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:56:59.810834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115901f-b46c-4aeb-ac2f-d7aa0e90228e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:59.854207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c60be0-0149-44e1-83e2-4323205b8e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:59.854691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c60be0-0149-44e1-83e2-4323205b8e48 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:59.854966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c60be0-0149-44e1-83e2-4323205b8e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:59.855348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c60be0-0149-44e1-83e2-4323205b8e48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:59.891666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839abc6-e347-4274-b34b-3df0cbf0bd0f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:56:59.893541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:59.911123 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:56:59.944866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b06ca8-49c1-4211-9571-2ebc8b5ba0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:56:59.946327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b06ca8-49c1-4211-9571-2ebc8b5ba0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:56:59.946947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b06ca8-49c1-4211-9571-2ebc8b5ba0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:56:59.947388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b06ca8-49c1-4211-9571-2ebc8b5ba0a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:56:59.957998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c60be0-0149-44e1-83e2-4323205b8e48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:56:59.960468 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c60be0-0149-44e1-83e2-4323205b8e48 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:56:59.964944 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b06ca8-49c1-4211-9571-2ebc8b5ba0a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-51b06ca8-49c1-4211-9571-2ebc8b5ba0a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:56:59.979778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:56:59.981201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:56:59.982151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:56:59.982151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:56:59.982151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:56:59.982151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:56:59.982151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:56:59.982151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:56:59.982151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:56:59.982151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:56:59.982151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:56:59.982151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:56:59.982151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:56:59.982151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:56:59.982151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 416/847] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:56:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:56:59.983366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b06ca8-49c1-4211-9571-2ebc8b5ba0a3 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:00.106713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 432/848] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:56:59 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:57:00.112983 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c60be0-0149-44e1-83e2-4323205b8e48 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:00.125774 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b06ca8-49c1-4211-9571-2ebc8b5ba0a3 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:00.170704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfd08b-37fb-44e2-9558-97077d19d165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:00.172464 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfd08b-37fb-44e2-9558-97077d19d165 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:00.172777 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfd08b-37fb-44e2-9558-97077d19d165 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:00.173143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfd08b-37fb-44e2-9558-97077d19d165 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:00.177674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfd08b-37fb-44e2-9558-97077d19d165 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:00.178012 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfd08b-37fb-44e2-9558-97077d19d165 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:00.180841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfd08b-37fb-44e2-9558-97077d19d165 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:00.183574 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:00.189979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 417/849] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:56:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:00.191512 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfd08b-37fb-44e2-9558-97077d19d165 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:00.255204 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:00.286709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c60be0-0149-44e1-83e2-4323205b8e48 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:00.368239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94dd6e-184a-4d94-bd28-272b1c302496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:00.369447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94dd6e-184a-4d94-bd28-272b1c302496 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:00.370013 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94dd6e-184a-4d94-bd28-272b1c302496 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:00.370578 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94dd6e-184a-4d94-bd28-272b1c302496 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:00.372631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94dd6e-184a-4d94-bd28-272b1c302496 None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:00.372979 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94dd6e-184a-4d94-bd28-272b1c302496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:00.375209 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94dd6e-184a-4d94-bd28-272b1c302496 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:00.387414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfd08b-37fb-44e2-9558-97077d19d165 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:00.387856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 433/850] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:57:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:00.394644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94dd6e-184a-4d94-bd28-272b1c302496 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:00.440301 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbf960-d8f5-4a34-92f7-942cf2811a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:00.441389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbf960-d8f5-4a34-92f7-942cf2811a0e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:00.441709 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbf960-d8f5-4a34-92f7-942cf2811a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:00.442455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbf960-d8f5-4a34-92f7-942cf2811a0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:00.455442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbf960-d8f5-4a34-92f7-942cf2811a0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:00.457009 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbf960-d8f5-4a34-92f7-942cf2811a0e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:00.477937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-43bbf960-d8f5-4a34-92f7-942cf2811a0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:00.477937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:00.477937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:00.477937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:00.477937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:00.477937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:00.477937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:00.477937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:00.477937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:00.477937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:00.477937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:00.477937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:00.477937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:00.477937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:00.477937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:00.478971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:00.479898 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:00.479898 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:00.479898 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:00.479898 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:00.482293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 418/851] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:00.491381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbf960-d8f5-4a34-92f7-942cf2811a0e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:00.573654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94dd6e-184a-4d94-bd28-272b1c302496 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:00.600875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:00.609570 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:00.617998 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbf960-d8f5-4a34-92f7-942cf2811a0e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:00.630818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22176777-2ee7-45f7-93cc-c6dfd7f09e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:00.631604 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22176777-2ee7-45f7-93cc-c6dfd7f09e1c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:00.631894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22176777-2ee7-45f7-93cc-c6dfd7f09e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:00.632219 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22176777-2ee7-45f7-93cc-c6dfd7f09e1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:00.645316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22176777-2ee7-45f7-93cc-c6dfd7f09e1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:00.647095 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22176777-2ee7-45f7-93cc-c6dfd7f09e1c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:00.659839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363af66-d474-4f89-94ce-125d828d7261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:00.660896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363af66-d474-4f89-94ce-125d828d7261 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:00.661390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363af66-d474-4f89-94ce-125d828d7261 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:00.661881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363af66-d474-4f89-94ce-125d828d7261 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:00.662810 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363af66-d474-4f89-94ce-125d828d7261 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:00.663299 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363af66-d474-4f89-94ce-125d828d7261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:00.664960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363af66-d474-4f89-94ce-125d828d7261 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:00.668411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-22176777-2ee7-45f7-93cc-c6dfd7f09e1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:00.668411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:00.668411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:00.668411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:00.668411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:00.668411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:00.668411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:00.668411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:00.668411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:00.668411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:00.668411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:00.668411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:00.668411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:00.668411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:00.668411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:00.669294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:00.670494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:00.670494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:00.670494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:00.670494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:00.670494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 434/852] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:00.681866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22176777-2ee7-45f7-93cc-c6dfd7f09e1c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:00.683187 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 419/853] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:00.689843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363af66-d474-4f89-94ce-125d828d7261 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:00.847340 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22176777-2ee7-45f7-93cc-c6dfd7f09e1c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:00.848692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363af66-d474-4f89-94ce-125d828d7261 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:00.895341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1675c-e753-410a-b923-f22c6f54ed78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:00.896069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1675c-e753-410a-b923-f22c6f54ed78 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:00.896380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1675c-e753-410a-b923-f22c6f54ed78 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:00.896682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1675c-e753-410a-b923-f22c6f54ed78 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:00.897453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1675c-e753-410a-b923-f22c6f54ed78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:00.897762 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1675c-e753-410a-b923-f22c6f54ed78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:00.899150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1675c-e753-410a-b923-f22c6f54ed78 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:00.900187 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c7b9d-819e-4f2f-b8c8-b5a2d2e5fef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:00.901403 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c7b9d-819e-4f2f-b8c8-b5a2d2e5fef2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:00.901859 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c7b9d-819e-4f2f-b8c8-b5a2d2e5fef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:00.902400 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c7b9d-819e-4f2f-b8c8-b5a2d2e5fef2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:00.913863 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c7b9d-819e-4f2f-b8c8-b5a2d2e5fef2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:00.915984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c7b9d-819e-4f2f-b8c8-b5a2d2e5fef2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:00.920085 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 435/854] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:00.928481 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1675c-e753-410a-b923-f22c6f54ed78 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:00.941111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-fd4c7b9d-819e-4f2f-b8c8-b5a2d2e5fef2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:00.941111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:00.941111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:00.941111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:00.941111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:00.941111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:00.941111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:00.941111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:00.941111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:00.941111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:00.941111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:00.941111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:00.941111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:57:00.941111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:57:00.941111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:00.942344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:00.943527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:57:00.943527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:57:00.943527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:57:00.943527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:57:00.943527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:57:00.943527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:57:00.943527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:00.943527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:00.951488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 420/855] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:57:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:00.954085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c7b9d-819e-4f2f-b8c8-b5a2d2e5fef2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:00.965672 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:00.972397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:01.087175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1675c-e753-410a-b923-f22c6f54ed78 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:01.122931 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c7b9d-819e-4f2f-b8c8-b5a2d2e5fef2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:01.140253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ac1da-6bb2-469e-bb9b-9bf1b10d07d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:01.141216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ac1da-6bb2-469e-bb9b-9bf1b10d07d3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:01.141442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ac1da-6bb2-469e-bb9b-9bf1b10d07d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:01.141741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ac1da-6bb2-469e-bb9b-9bf1b10d07d3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:01.151259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ac1da-6bb2-469e-bb9b-9bf1b10d07d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:01.152911 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ac1da-6bb2-469e-bb9b-9bf1b10d07d3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:01.173310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5489f-40d3-4bc1-9c38-917e866bc0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:01.174854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5489f-40d3-4bc1-9c38-917e866bc0fd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:01.175357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5489f-40d3-4bc1-9c38-917e866bc0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:01.175847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c5489f-40d3-4bc1-9c38-917e866bc0fd None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-e88ac1da-6bb2-469e-bb9b-9bf1b10d07d3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:57:01.176512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:01.177401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:01.178330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:01.178330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:01.178330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:57:01.178330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:57:01.178330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:57:01.178330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:57:01.178330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:57:01.178330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:57:01.178330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:57:01.178330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:01.179891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5489f-40d3-4bc1-9c38-917e866bc0fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:01.180456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5489f-40d3-4bc1-9c38-917e866bc0fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:01.182172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c5489f-40d3-4bc1-9c38-917e866bc0fd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:01.183119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 436/856] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jul 21 08:57:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:01.184969 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ac1da-6bb2-469e-bb9b-9bf1b10d07d3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:01.201340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 421/857] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:57:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:01.208352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5489f-40d3-4bc1-9c38-917e866bc0fd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:01.308751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:01.319556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:01.334457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ac1da-6bb2-469e-bb9b-9bf1b10d07d3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:01.353915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c5489f-40d3-4bc1-9c38-917e866bc0fd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:01.384598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856fed2-d652-41ff-82a8-6591375a29d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:01.386623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856fed2-d652-41ff-82a8-6591375a29d4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:01.387287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856fed2-d652-41ff-82a8-6591375a29d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:01.388282 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4856fed2-d652-41ff-82a8-6591375a29d4 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:01.389172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4856fed2-d652-41ff-82a8-6591375a29d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:01.389485 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4856fed2-d652-41ff-82a8-6591375a29d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:01.391516 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4856fed2-d652-41ff-82a8-6591375a29d4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:01.398534 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843a694-ce38-4797-ba24-e74c04be7caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:01.399443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a694-ce38-4797-ba24-e74c04be7caa None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:01.399825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a694-ce38-4797-ba24-e74c04be7caa None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:01.400190 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a694-ce38-4797-ba24-e74c04be7caa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:01.401501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843a694-ce38-4797-ba24-e74c04be7caa None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=061def74ec7f42ce9a174e0944bf499f) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:01.401626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843a694-ce38-4797-ba24-e74c04be7caa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:01.403345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843a694-ce38-4797-ba24-e74c04be7caa None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:01.419945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 422/858] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:57:01 2022] GET /identity/v3/role_assignments?user.id=061def74ec7f42ce9a174e0944bf499f&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=000e942517f2469ba36d2e65978bc314 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:01.419945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e62dee-f1ee-447d-b972-88d4597cb213 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:01.419945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e62dee-f1ee-447d-b972-88d4597cb213 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:01.420667 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 437/859] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jul 21 08:57:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:01.420918 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e62dee-f1ee-447d-b972-88d4597cb213 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:01.420918 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 423/860] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:01.424262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856fed2-d652-41ff-82a8-6591375a29d4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:01.424863 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd43dba-f31e-4eae-95fb-14733a13ea28 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:01.425331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd43dba-f31e-4eae-95fb-14733a13ea28 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:01.425738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd43dba-f31e-4eae-95fb-14733a13ea28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:01.448157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-7dd43dba-f31e-4eae-95fb-14733a13ea28 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:01.449437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7dd43dba-f31e-4eae-95fb-14733a13ea28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:01.550389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 424/861] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:01.579350 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd43dba-f31e-4eae-95fb-14733a13ea28 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:01.584088 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4856fed2-d652-41ff-82a8-6591375a29d4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:01.623531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b37dc5-9cff-4627-90b5-379e62990219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:01.624996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b37dc5-9cff-4627-90b5-379e62990219 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:01.625193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b37dc5-9cff-4627-90b5-379e62990219 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:01.625664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b37dc5-9cff-4627-90b5-379e62990219 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:01.647888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:01.678417 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:01.744564 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b37dc5-9cff-4627-90b5-379e62990219 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:01.746053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd43dba-f31e-4eae-95fb-14733a13ea28 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:01.746963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b37dc5-9cff-4627-90b5-379e62990219 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:01.787448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2da8b-7d12-4c6d-9f05-5948d8a28903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:01.788379 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2da8b-7d12-4c6d-9f05-5948d8a28903 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:01.788668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2da8b-7d12-4c6d-9f05-5948d8a28903 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:01.788993 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2da8b-7d12-4c6d-9f05-5948d8a28903 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:01.789683 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2da8b-7d12-4c6d-9f05-5948d8a28903 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:01.791262 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2da8b-7d12-4c6d-9f05-5948d8a28903 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-8df2da8b-7d12-4c6d-9f05-5948d8a28903 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:01.806243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:01.807268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:01.808155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:01.808155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:01.808155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:01.808155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:01.808155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:01.808155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:01.808155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:01.808155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:01.808155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:01.808155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:01.808155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:01.808155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:01.808155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:01.808155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:57:01.808155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:01.812566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 425/862] 10.222.0.13 () {56 vars in 1132 bytes} [Thu Jul 21 08:57:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:01.814902 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2da8b-7d12-4c6d-9f05-5948d8a28903 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:01.905205 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 438/863] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:57:01 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:57:01.915187 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b37dc5-9cff-4627-90b5-379e62990219 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:01.945625 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2da8b-7d12-4c6d-9f05-5948d8a28903 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:01.980179 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:01.988426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:01.990476 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d56cfa-0c12-4811-9bbc-9fb376df33eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:01.991176 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d56cfa-0c12-4811-9bbc-9fb376df33eb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:01.991460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d56cfa-0c12-4811-9bbc-9fb376df33eb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:01.992027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d56cfa-0c12-4811-9bbc-9fb376df33eb None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:01.992776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d56cfa-0c12-4811-9bbc-9fb376df33eb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:01.993467 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d56cfa-0c12-4811-9bbc-9fb376df33eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:01.995612 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d56cfa-0c12-4811-9bbc-9fb376df33eb None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:02.006803 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 426/864] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jul 21 08:57:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:02.007895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d56cfa-0c12-4811-9bbc-9fb376df33eb None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:02.040459 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b37dc5-9cff-4627-90b5-379e62990219 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:02.079842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93e131-5bb2-44d8-809c-68f01408dd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:02.081047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93e131-5bb2-44d8-809c-68f01408dd84 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:02.081540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93e131-5bb2-44d8-809c-68f01408dd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:02.082049 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93e131-5bb2-44d8-809c-68f01408dd84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:02.094041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc93e131-5bb2-44d8-809c-68f01408dd84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:02.096061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc93e131-5bb2-44d8-809c-68f01408dd84 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:02.119106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-bc93e131-5bb2-44d8-809c-68f01408dd84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:02.119106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:02.119106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:02.119106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:02.119106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:02.119106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:02.119106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:02.119106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:02.119106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:02.119106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:02.119106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:02.119106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:02.119106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:02.119106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:02.119106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:02.120116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:02.120995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:02.120995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:02.120995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:02.120995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:02.123457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 439/865] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:02.131967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93e131-5bb2-44d8-809c-68f01408dd84 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:02.146559 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d56cfa-0c12-4811-9bbc-9fb376df33eb None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:02.184302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c7dc6-4fcc-4058-933c-b3d351d0dcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:02.185084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c7dc6-4fcc-4058-933c-b3d351d0dcff None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:02.185364 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c7dc6-4fcc-4058-933c-b3d351d0dcff None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:02.185642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c7dc6-4fcc-4058-933c-b3d351d0dcff None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:02.186350 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c7dc6-4fcc-4058-933c-b3d351d0dcff None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:02.186649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c7dc6-4fcc-4058-933c-b3d351d0dcff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:02.188712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c7dc6-4fcc-4058-933c-b3d351d0dcff None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:02.200034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 427/866] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jul 21 08:57:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:02.254352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:02.259079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93e131-5bb2-44d8-809c-68f01408dd84 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:02.301570 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3866d33-6ea2-4062-b07a-294f7e9091a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:02.302248 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3866d33-6ea2-4062-b07a-294f7e9091a7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:02.302529 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3866d33-6ea2-4062-b07a-294f7e9091a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:02.302802 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3866d33-6ea2-4062-b07a-294f7e9091a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:02.303536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3866d33-6ea2-4062-b07a-294f7e9091a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:02.303837 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3866d33-6ea2-4062-b07a-294f7e9091a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:02.305267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3866d33-6ea2-4062-b07a-294f7e9091a7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:02.323119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 440/867] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:02.324613 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c7dc6-4fcc-4058-933c-b3d351d0dcff None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:02.392814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:02.442799 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c7dc6-4fcc-4058-933c-b3d351d0dcff None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:02.482996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c576ec-0fcb-48e5-9b2b-40aa843abe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:02.484514 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c576ec-0fcb-48e5-9b2b-40aa843abe0f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:02.484794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c576ec-0fcb-48e5-9b2b-40aa843abe0f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:02.485072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c576ec-0fcb-48e5-9b2b-40aa843abe0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:02.492313 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c576ec-0fcb-48e5-9b2b-40aa843abe0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:02.493760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c576ec-0fcb-48e5-9b2b-40aa843abe0f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:02.508272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-56c576ec-0fcb-48e5-9b2b-40aa843abe0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:02.508272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:02.508272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:02.508272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:02.508272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:02.508272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:02.508272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:02.508272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:02.508272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:02.508272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:02.508272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:02.508272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:02.508272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:57:02.508272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:57:02.508272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:02.509211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:02.510330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:57:02.510330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:57:02.510330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:57:02.510330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:57:02.510330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:57:02.510330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:57:02.510330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:02.510330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:02.510330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 428/868] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:57:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:02.517500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3866d33-6ea2-4062-b07a-294f7e9091a7 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:02.520447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fa091-13ad-46d3-acb8-11ca2469f598 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:02.520740 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fa091-13ad-46d3-acb8-11ca2469f598 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:02.521023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fa091-13ad-46d3-acb8-11ca2469f598 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:02.522622 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 429/869] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:02.527592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052fa12-64c6-4d7f-8b95-0e9b8a0269bc None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:02.527882 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052fa12-64c6-4d7f-8b95-0e9b8a0269bc None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:02.528187 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052fa12-64c6-4d7f-8b95-0e9b8a0269bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:02.548762 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d052fa12-64c6-4d7f-8b95-0e9b8a0269bc None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:02.550257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d052fa12-64c6-4d7f-8b95-0e9b8a0269bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:02.632450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3866d33-6ea2-4062-b07a-294f7e9091a7 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:02.656015 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 430/870] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:02.673523 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbd26e-d644-47ea-8027-1393c6a0233c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:02.674264 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052fa12-64c6-4d7f-8b95-0e9b8a0269bc None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:02.674791 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbd26e-d644-47ea-8027-1393c6a0233c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:02.675290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbd26e-d644-47ea-8027-1393c6a0233c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:02.675828 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbd26e-d644-47ea-8027-1393c6a0233c None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:02.676853 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbd26e-d644-47ea-8027-1393c6a0233c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:02.677383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbd26e-d644-47ea-8027-1393c6a0233c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:02.678901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbd26e-d644-47ea-8027-1393c6a0233c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:02.694038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 441/871] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:57:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:02.700349 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbd26e-d644-47ea-8027-1393c6a0233c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:02.726151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:02.746536 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:02.812768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052fa12-64c6-4d7f-8b95-0e9b8a0269bc None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:02.823965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbd26e-d644-47ea-8027-1393c6a0233c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:02.852656 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f377fc-aa1e-4399-a673-298dccec7490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:02.853476 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f377fc-aa1e-4399-a673-298dccec7490 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:02.853764 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f377fc-aa1e-4399-a673-298dccec7490 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:02.854038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f377fc-aa1e-4399-a673-298dccec7490 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:02.854845 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f377fc-aa1e-4399-a673-298dccec7490 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:02.856271 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f377fc-aa1e-4399-a673-298dccec7490 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:02.863067 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be99092-7de4-4663-9cce-0d7739b7b2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:02.863982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be99092-7de4-4663-9cce-0d7739b7b2ea None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:02.864355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be99092-7de4-4663-9cce-0d7739b7b2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:02.864566 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be99092-7de4-4663-9cce-0d7739b7b2ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:02.865378 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be99092-7de4-4663-9cce-0d7739b7b2ea None admin] RBAC: Adding query filter params (role.id=8511cf3e5d2c4acdae6d0ffda5e72c63, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=257450db238d4efda783e30cbcd23d4f) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:02.865730 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be99092-7de4-4663-9cce-0d7739b7b2ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:02.867526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be99092-7de4-4663-9cce-0d7739b7b2ea None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:02.873712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 431/872] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jul 21 08:57:02 2022] POST /identity/v3/services => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:57:02.875983 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 442/873] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:57:02 2022] GET /identity/v3/role_assignments?user.id=257450db238d4efda783e30cbcd23d4f&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:03.699196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ec0ce-6c75-476c-9c56-aeedd745cfcb None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:03.699764 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ec0ce-6c75-476c-9c56-aeedd745cfcb None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:03.699764 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ec0ce-6c75-476c-9c56-aeedd745cfcb None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:03.701412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 432/874] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:03.707731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647e095-c229-48ad-aebf-6dac15ff077f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:03.708248 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647e095-c229-48ad-aebf-6dac15ff077f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:03.708324 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647e095-c229-48ad-aebf-6dac15ff077f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:03.727086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-8647e095-c229-48ad-aebf-6dac15ff077f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:03.728763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8647e095-c229-48ad-aebf-6dac15ff077f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:03.810954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 443/875] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:03 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:03.835729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ec0ce-6c75-476c-9c56-aeedd745cfcb None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:03.948211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ec0ce-6c75-476c-9c56-aeedd745cfcb None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:03.950611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:03.985391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3770c0-e6f5-4860-bfba-e040d0d09746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:03.986192 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3770c0-e6f5-4860-bfba-e040d0d09746 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:03.986472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3770c0-e6f5-4860-bfba-e040d0d09746 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:03.986745 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3770c0-e6f5-4860-bfba-e040d0d09746 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:03.987396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3770c0-e6f5-4860-bfba-e040d0d09746 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:03.988789 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3770c0-e6f5-4860-bfba-e040d0d09746 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-0f3770c0-e6f5-4860-bfba-e040d0d09746 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:04.001654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:04.002696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:04.003527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:04.003527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:04.003527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:04.003527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:04.003527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:04.003527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:04.003527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:04.003527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:04.003527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:04.003527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:04.003527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:04.003527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:04.003527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:57:04.003527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:04.003527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 433/876] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jul 21 08:57:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:04.009320 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647e095-c229-48ad-aebf-6dac15ff077f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:04.021827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:04.119447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647e095-c229-48ad-aebf-6dac15ff077f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:04.161830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9eb8a-e232-49e4-a496-f9e2214e7b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:04.162784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9eb8a-e232-49e4-a496-f9e2214e7b09 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:04.163250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9eb8a-e232-49e4-a496-f9e2214e7b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:04.163671 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9eb8a-e232-49e4-a496-f9e2214e7b09 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:04.164593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9eb8a-e232-49e4-a496-f9e2214e7b09 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:04.165053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9eb8a-e232-49e4-a496-f9e2214e7b09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:04.166675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9eb8a-e232-49e4-a496-f9e2214e7b09 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:04.172987 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 444/877] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jul 21 08:57:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:04.179946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3770c0-e6f5-4860-bfba-e040d0d09746 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:04.287895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3770c0-e6f5-4860-bfba-e040d0d09746 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:04.322145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f060a5-418b-48d3-8d92-b5f16286bb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:04.322809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f060a5-418b-48d3-8d92-b5f16286bb6f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:04.323090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f060a5-418b-48d3-8d92-b5f16286bb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:04.323358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f060a5-418b-48d3-8d92-b5f16286bb6f None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:04.324077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f060a5-418b-48d3-8d92-b5f16286bb6f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:04.324381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f060a5-418b-48d3-8d92-b5f16286bb6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:04.325755 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f060a5-418b-48d3-8d92-b5f16286bb6f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:04.331179 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 434/878] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jul 21 08:57:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:04.408656 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489a3e2-4d2c-45cc-aa3f-facc4676594b None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:04.408984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489a3e2-4d2c-45cc-aa3f-facc4676594b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:04.409283 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489a3e2-4d2c-45cc-aa3f-facc4676594b None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:04.411293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 445/879] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:04.418653 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da57633-64d1-421f-b375-ae17a24e620a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:04.419112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da57633-64d1-421f-b375-ae17a24e620a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:04.419560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da57633-64d1-421f-b375-ae17a24e620a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:04.446619 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6da57633-64d1-421f-b375-ae17a24e620a None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:04.447880 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6da57633-64d1-421f-b375-ae17a24e620a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:04.480779 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:04.531760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 435/880] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:04 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:04.554385 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489a3e2-4d2c-45cc-aa3f-facc4676594b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:04.645071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:04.673995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489a3e2-4d2c-45cc-aa3f-facc4676594b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:04.720332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df416eaf-da7f-4a75-9e77-3043dadb518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:04.721092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df416eaf-da7f-4a75-9e77-3043dadb518e None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:04.721293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df416eaf-da7f-4a75-9e77-3043dadb518e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:04.721571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df416eaf-da7f-4a75-9e77-3043dadb518e None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:04.722330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df416eaf-da7f-4a75-9e77-3043dadb518e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:04.723731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df416eaf-da7f-4a75-9e77-3043dadb518e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:04.741242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 446/881] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jul 21 08:57:04 2022] POST /identity/v3/services => generated 269 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:57:05.132582 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62886829-1c18-4884-847c-df019b566e11 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:05.133631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62886829-1c18-4884-847c-df019b566e11 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:05.134124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62886829-1c18-4884-847c-df019b566e11 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:05.136007 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 436/882] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:05.141881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316dc375-c4c2-43ac-a738-472c711d285d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:05.142450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316dc375-c4c2-43ac-a738-472c711d285d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:05.142985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316dc375-c4c2-43ac-a738-472c711d285d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:05.160870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-316dc375-c4c2-43ac-a738-472c711d285d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:05.162402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-316dc375-c4c2-43ac-a738-472c711d285d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:05.172674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca12357-e8bd-44af-9519-068ffe491d97 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:05.172973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca12357-e8bd-44af-9519-068ffe491d97 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:05.173237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca12357-e8bd-44af-9519-068ffe491d97 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:05.175636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 437/883] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:05.181543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed712ca4-d778-472b-babe-e8661cc0ed31 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:05.182025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed712ca4-d778-472b-babe-e8661cc0ed31 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:05.182472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed712ca4-d778-472b-babe-e8661cc0ed31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:05.210844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ed712ca4-d778-472b-babe-e8661cc0ed31 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:05.212866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ed712ca4-d778-472b-babe-e8661cc0ed31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:05.251018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 447/884] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:05.274120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316dc375-c4c2-43ac-a738-472c711d285d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:05.310559 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 438/885] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:05.334291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed712ca4-d778-472b-babe-e8661cc0ed31 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:05.343781 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:05.397486 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316dc375-c4c2-43ac-a738-472c711d285d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:05.433500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:05.443826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed712ca4-d778-472b-babe-e8661cc0ed31 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:05.452615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58af084-a960-4066-8705-276efccf6e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:05.453926 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58af084-a960-4066-8705-276efccf6e74 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:05.454497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58af084-a960-4066-8705-276efccf6e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:05.454926 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58af084-a960-4066-8705-276efccf6e74 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:05.455886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58af084-a960-4066-8705-276efccf6e74 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:05.457467 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58af084-a960-4066-8705-276efccf6e74 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-f58af084-a960-4066-8705-276efccf6e74 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:05.473909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:05.475138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:05.476059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:05.476059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:05.476059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:05.476059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:05.476059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:05.476059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:05.476059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:05.476059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:05.476059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:05.476059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:05.476059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:05.476059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:05.476059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:57:05.476059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:05.480121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 448/886] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:57:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:05.481120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58af084-a960-4066-8705-276efccf6e74 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:05.482679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fcc87-b53f-4692-80ba-cf034e7f1169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:05.483496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fcc87-b53f-4692-80ba-cf034e7f1169 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:05.484386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fcc87-b53f-4692-80ba-cf034e7f1169 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:05.484823 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fcc87-b53f-4692-80ba-cf034e7f1169 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:05.498812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fcc87-b53f-4692-80ba-cf034e7f1169 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-106fcc87-b53f-4692-80ba-cf034e7f1169 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:57:05.508950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:05.510103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:05.511633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:05.511633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:05.511633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:05.511633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:05.511633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:57:05.511633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:57:05.511633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:57:05.511633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:57:05.511633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:57:05.511633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:57:05.511633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:57:05.511633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:05.518071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 439/887] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jul 21 08:57:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:05.519010 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fcc87-b53f-4692-80ba-cf034e7f1169 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:05.613630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58af084-a960-4066-8705-276efccf6e74 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:05.642311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:05.650160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fcc87-b53f-4692-80ba-cf034e7f1169 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:05.654966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb1f00-01c8-4ad7-9be8-77559074901c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:05.655789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb1f00-01c8-4ad7-9be8-77559074901c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:05.656740 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb1f00-01c8-4ad7-9be8-77559074901c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:05.657103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb1f00-01c8-4ad7-9be8-77559074901c None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:05.657894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb1f00-01c8-4ad7-9be8-77559074901c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:05.658207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb1f00-01c8-4ad7-9be8-77559074901c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:05.659602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb1f00-01c8-4ad7-9be8-77559074901c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:05.669737 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 449/888] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:57:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:05.670596 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb1f00-01c8-4ad7-9be8-77559074901c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:05.695676 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c188619-3069-4ea2-930e-27f77c289012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:05.696873 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c188619-3069-4ea2-930e-27f77c289012 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:05.697337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c188619-3069-4ea2-930e-27f77c289012 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:05.698134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c188619-3069-4ea2-930e-27f77c289012 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:05.699319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c188619-3069-4ea2-930e-27f77c289012 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:05.700105 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c188619-3069-4ea2-930e-27f77c289012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:05.702367 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c188619-3069-4ea2-930e-27f77c289012 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:05.714639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 440/889] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jul 21 08:57:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:05.721105 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c188619-3069-4ea2-930e-27f77c289012 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:05.742104 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:05.788933 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb1f00-01c8-4ad7-9be8-77559074901c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:05.826025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba77887-4bcf-43ce-af20-84a076122552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:05.826794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba77887-4bcf-43ce-af20-84a076122552 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:05.827165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba77887-4bcf-43ce-af20-84a076122552 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:05.827535 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba77887-4bcf-43ce-af20-84a076122552 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:05.828453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba77887-4bcf-43ce-af20-84a076122552 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:05.828831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba77887-4bcf-43ce-af20-84a076122552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:05.830361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba77887-4bcf-43ce-af20-84a076122552 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:05.836425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 450/890] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:57:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:05.838361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c188619-3069-4ea2-930e-27f77c289012 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:05.842259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba77887-4bcf-43ce-af20-84a076122552 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:05.877879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a7341-d6eb-4b99-a9f4-9aadb4e504b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:05.878625 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a7341-d6eb-4b99-a9f4-9aadb4e504b0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:05.878913 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a7341-d6eb-4b99-a9f4-9aadb4e504b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:05.879189 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a7341-d6eb-4b99-a9f4-9aadb4e504b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:05.891210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a7341-d6eb-4b99-a9f4-9aadb4e504b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:05.892850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a7341-d6eb-4b99-a9f4-9aadb4e504b0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:05.904840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:05.915265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-ab2a7341-d6eb-4b99-a9f4-9aadb4e504b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:05.915265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:05.915265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:05.915265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:05.915265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:05.915265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:05.915265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:05.915265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:05.915265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:05.915265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:05.915265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:05.915265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:05.915265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:05.915265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:05.915265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:05.916252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:05.917136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:05.917136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:05.917136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:05.917136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:05.917136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 441/891] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:05.923177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a7341-d6eb-4b99-a9f4-9aadb4e504b0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:05.966640 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba77887-4bcf-43ce-af20-84a076122552 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:05.994690 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:06.016495 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb0ad9-0bc6-48f7-9104-5cb4810887d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:06.017539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb0ad9-0bc6-48f7-9104-5cb4810887d7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:06.018065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb0ad9-0bc6-48f7-9104-5cb4810887d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:06.018634 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb0ad9-0bc6-48f7-9104-5cb4810887d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:06.019614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb0ad9-0bc6-48f7-9104-5cb4810887d7 None admin] RBAC: Adding query filter params (interface=public, service_id=e8a2fd9eca0a45dcadbaa9c8540802de, region_id=RegionOne) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:06.020143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb0ad9-0bc6-48f7-9104-5cb4810887d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:06.021837 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb0ad9-0bc6-48f7-9104-5cb4810887d7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:06.028327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 451/892] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jul 21 08:57:05 2022] GET /identity/v3/endpoints?service_id=e8a2fd9eca0a45dcadbaa9c8540802de&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:06.034376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb0ad9-0bc6-48f7-9104-5cb4810887d7 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:06.036623 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a7341-d6eb-4b99-a9f4-9aadb4e504b0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:06.078441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9d719-88eb-4ffa-9d16-60c3416936ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:06.079367 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9d719-88eb-4ffa-9d16-60c3416936ef None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:06.079656 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9d719-88eb-4ffa-9d16-60c3416936ef None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:06.079951 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9d719-88eb-4ffa-9d16-60c3416936ef None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:06.080668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9d719-88eb-4ffa-9d16-60c3416936ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:06.080976 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9d719-88eb-4ffa-9d16-60c3416936ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:06.082505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9d719-88eb-4ffa-9d16-60c3416936ef None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:06.106614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 442/893] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:06.108286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9d719-88eb-4ffa-9d16-60c3416936ef None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:06.167389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb0ad9-0bc6-48f7-9104-5cb4810887d7 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:06.190901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:06.214686 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cae7da-4053-4abe-8403-58cfccfc89c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:06.215883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cae7da-4053-4abe-8403-58cfccfc89c4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:06.216757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cae7da-4053-4abe-8403-58cfccfc89c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:06.217510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cae7da-4053-4abe-8403-58cfccfc89c4 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:06.219077 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cae7da-4053-4abe-8403-58cfccfc89c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:06.221333 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cae7da-4053-4abe-8403-58cfccfc89c4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:06.222679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9d719-88eb-4ffa-9d16-60c3416936ef None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:06.236585 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 452/894] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jul 21 08:57:06 2022] GET /identity/v3/services => generated 1268 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:57:06.258716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:06.270918 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cd65a-a004-4c99-96e3-8a2b8b2891ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:06.271756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cd65a-a004-4c99-96e3-8a2b8b2891ef None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:06.272077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cd65a-a004-4c99-96e3-8a2b8b2891ef None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:06.272353 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cd65a-a004-4c99-96e3-8a2b8b2891ef None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:06.281441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cd65a-a004-4c99-96e3-8a2b8b2891ef None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:06.283075 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cd65a-a004-4c99-96e3-8a2b8b2891ef None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-267cd65a-a004-4c99-96e3-8a2b8b2891ef None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:57:06.308960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:06.309984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:06.310923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:06.310923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:06.310923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:57:06.310923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:57:06.310923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:57:06.310923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:57:06.310923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:57:06.310923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:57:06.310923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:57:06.310923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:06.310923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 443/895] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jul 21 08:57:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:06.324869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cae7da-4053-4abe-8403-58cfccfc89c4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:06.461479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cae7da-4053-4abe-8403-58cfccfc89c4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:06.498361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0247f-f431-4f16-b391-77b73e70fcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:06.499086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0247f-f431-4f16-b391-77b73e70fcdb None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:06.499369 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0247f-f431-4f16-b391-77b73e70fcdb None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:06.500337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0247f-f431-4f16-b391-77b73e70fcdb None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:06.501375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0247f-f431-4f16-b391-77b73e70fcdb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:06.501682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0247f-f431-4f16-b391-77b73e70fcdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:06.503130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0247f-f431-4f16-b391-77b73e70fcdb None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:06.522597 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 453/896] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jul 21 08:57:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:06.528392 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cd65a-a004-4c99-96e3-8a2b8b2891ef None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:06.637234 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cd65a-a004-4c99-96e3-8a2b8b2891ef None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:06.675321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de638c4-b374-4a15-ac80-0fb4d7c27dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:06.676382 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de638c4-b374-4a15-ac80-0fb4d7c27dfe None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:06.676905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de638c4-b374-4a15-ac80-0fb4d7c27dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:06.677389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de638c4-b374-4a15-ac80-0fb4d7c27dfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:06.738148 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:06.752671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de638c4-b374-4a15-ac80-0fb4d7c27dfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:06.753801 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbac2a3-34e9-4f95-8946-96a6e5d75bef None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:06.754094 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbac2a3-34e9-4f95-8946-96a6e5d75bef None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:06.754196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de638c4-b374-4a15-ac80-0fb4d7c27dfe None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:06.754375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbac2a3-34e9-4f95-8946-96a6e5d75bef None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:06.759875 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 454/897] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:57:06.761191 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976b8d6-75e7-4cc9-a64d-a8483c7841f8 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:06.761479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976b8d6-75e7-4cc9-a64d-a8483c7841f8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:06.761759 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976b8d6-75e7-4cc9-a64d-a8483c7841f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:06.781870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3976b8d6-75e7-4cc9-a64d-a8483c7841f8 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:06.783435 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3976b8d6-75e7-4cc9-a64d-a8483c7841f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:06.832097 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:06.876015 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 444/898] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:57:06 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:57:06.877636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9be05-734a-4911-ba70-907ff0acc183 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:06.877932 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9be05-734a-4911-ba70-907ff0acc183 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:06.878214 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9be05-734a-4911-ba70-907ff0acc183 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:06.879746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 445/899] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:06.881540 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9be05-734a-4911-ba70-907ff0acc183 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:06.886198 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 455/900] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:06.888785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c3b89-7bd4-4cd7-8655-263a72735613 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:06.889077 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c3b89-7bd4-4cd7-8655-263a72735613 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:06.889353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c3b89-7bd4-4cd7-8655-263a72735613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:06.910845 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-776c3b89-7bd4-4cd7-8655-263a72735613 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:06.912550 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-776c3b89-7bd4-4cd7-8655-263a72735613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:06.992014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:06.992189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 456/901] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:06.992189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c3b89-7bd4-4cd7-8655-263a72735613 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:07.005194 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9be05-734a-4911-ba70-907ff0acc183 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:07.042306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1355583-a486-4b25-b025-2da4e35a4098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:07.043452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1355583-a486-4b25-b025-2da4e35a4098 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:07.043735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1355583-a486-4b25-b025-2da4e35a4098 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:07.044040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1355583-a486-4b25-b025-2da4e35a4098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:07.055642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1355583-a486-4b25-b025-2da4e35a4098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:07.057031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1355583-a486-4b25-b025-2da4e35a4098 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:07.075947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b1355583-a486-4b25-b025-2da4e35a4098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:07.075947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:07.075947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:07.075947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:07.075947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:07.075947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:07.075947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:07.075947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:07.075947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:07.075947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:07.075947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:07.075947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:07.075947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:07.075947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:07.075947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:07.076940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:07.078552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:07.078552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:07.078552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:07.078552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:07.078552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 446/902] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:07.078852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1355583-a486-4b25-b025-2da4e35a4098 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:07.106423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:07.110700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c3b89-7bd4-4cd7-8655-263a72735613 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:07.148902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9488b50-6ca5-4dde-b21b-20abf0a64296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:07.149574 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9488b50-6ca5-4dde-b21b-20abf0a64296 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:07.149871 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9488b50-6ca5-4dde-b21b-20abf0a64296 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:07.150128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9488b50-6ca5-4dde-b21b-20abf0a64296 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:07.150865 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9488b50-6ca5-4dde-b21b-20abf0a64296 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:07.152297 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9488b50-6ca5-4dde-b21b-20abf0a64296 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:07.167091 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 457/903] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jul 21 08:57:06 2022] POST /identity/v3/services => generated 241 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:57:07.168197 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9488b50-6ca5-4dde-b21b-20abf0a64296 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:07.196132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1355583-a486-4b25-b025-2da4e35a4098 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:07.248081 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198729c2-a3f4-4ef4-8d6a-75db1905e058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:07.249832 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198729c2-a3f4-4ef4-8d6a-75db1905e058 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:07.250460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198729c2-a3f4-4ef4-8d6a-75db1905e058 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:07.251017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198729c2-a3f4-4ef4-8d6a-75db1905e058 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:07.252046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198729c2-a3f4-4ef4-8d6a-75db1905e058 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:07.254004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198729c2-a3f4-4ef4-8d6a-75db1905e058 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:07.257319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-198729c2-a3f4-4ef4-8d6a-75db1905e058 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:07.273871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:07.274985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:07.275846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:07.275846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:07.275846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:07.275846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:07.275846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:07.275846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:07.275846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:07.275846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:07.275846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:07.275846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:07.275846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:07.275846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:07.275846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:07.275846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:57:07.275846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:07.279231 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 447/904] 10.222.0.13 () {56 vars in 1132 bytes} [Thu Jul 21 08:57:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:07.287463 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198729c2-a3f4-4ef4-8d6a-75db1905e058 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:07.325241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9488b50-6ca5-4dde-b21b-20abf0a64296 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:07.367403 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397f5f0-1553-486c-8d49-dd0b8c8164ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:07.368116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397f5f0-1553-486c-8d49-dd0b8c8164ec None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:07.368405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397f5f0-1553-486c-8d49-dd0b8c8164ec None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:07.368681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397f5f0-1553-486c-8d49-dd0b8c8164ec None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:07.369417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397f5f0-1553-486c-8d49-dd0b8c8164ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:07.369727 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397f5f0-1553-486c-8d49-dd0b8c8164ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:07.371130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397f5f0-1553-486c-8d49-dd0b8c8164ec None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:07.385143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 458/905] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:07.391060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397f5f0-1553-486c-8d49-dd0b8c8164ec None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:07.416167 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:07.431418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198729c2-a3f4-4ef4-8d6a-75db1905e058 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:07.476989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841354d5-ce79-4b92-af4f-0fe3b21bd0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:07.477825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841354d5-ce79-4b92-af4f-0fe3b21bd0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:07.478116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841354d5-ce79-4b92-af4f-0fe3b21bd0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:07.478399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841354d5-ce79-4b92-af4f-0fe3b21bd0c4 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:07.479142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841354d5-ce79-4b92-af4f-0fe3b21bd0c4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:07.479456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841354d5-ce79-4b92-af4f-0fe3b21bd0c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:07.481593 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841354d5-ce79-4b92-af4f-0fe3b21bd0c4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:07.492723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 448/906] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jul 21 08:57:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:07.494707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841354d5-ce79-4b92-af4f-0fe3b21bd0c4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:07.533903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397f5f0-1553-486c-8d49-dd0b8c8164ec None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:07.570817 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c677ef-ec9e-44c1-a76f-ae9db980ef4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:07.571644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c677ef-ec9e-44c1-a76f-ae9db980ef4c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:07.571942 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c677ef-ec9e-44c1-a76f-ae9db980ef4c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:07.572243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c677ef-ec9e-44c1-a76f-ae9db980ef4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:07.580373 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c677ef-ec9e-44c1-a76f-ae9db980ef4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:07.581314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:07.582093 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c677ef-ec9e-44c1-a76f-ae9db980ef4c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:07.597062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-d0c677ef-ec9e-44c1-a76f-ae9db980ef4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:07.597062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:07.597062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:07.597062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:07.597062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:07.597062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:07.597062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:07.597062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:07.597062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:07.597062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:07.597062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:07.597062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:07.597062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:57:07.597062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:57:07.597062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:07.598008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:07.598902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:57:07.598902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:57:07.598902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:57:07.598902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:57:07.598902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:57:07.598902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:57:07.598902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:07.598902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:07.603266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 459/907] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:57:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:07.604617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c677ef-ec9e-44c1-a76f-ae9db980ef4c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:07.629256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841354d5-ce79-4b92-af4f-0fe3b21bd0c4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:07.672313 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1ed4f-e106-496b-aebd-f157ce9a32e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:07.673606 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1ed4f-e106-496b-aebd-f157ce9a32e3 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:07.674160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1ed4f-e106-496b-aebd-f157ce9a32e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:07.674691 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1ed4f-e106-496b-aebd-f157ce9a32e3 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:07.675606 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1ed4f-e106-496b-aebd-f157ce9a32e3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:07.676161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1ed4f-e106-496b-aebd-f157ce9a32e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:07.677848 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1ed4f-e106-496b-aebd-f157ce9a32e3 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:07.688930 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 449/908] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jul 21 08:57:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:07.691247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1ed4f-e106-496b-aebd-f157ce9a32e3 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:07.692934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:07.725383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c677ef-ec9e-44c1-a76f-ae9db980ef4c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:07.773865 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51317bc-3f67-40f1-9bcd-3cfa8cd1dd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:07.776001 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51317bc-3f67-40f1-9bcd-3cfa8cd1dd96 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:07.776477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51317bc-3f67-40f1-9bcd-3cfa8cd1dd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:07.776940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51317bc-3f67-40f1-9bcd-3cfa8cd1dd96 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:07.777944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51317bc-3f67-40f1-9bcd-3cfa8cd1dd96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:07.778449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51317bc-3f67-40f1-9bcd-3cfa8cd1dd96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:07.780119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51317bc-3f67-40f1-9bcd-3cfa8cd1dd96 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:07.802046 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 460/909] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:57:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:07.804376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51317bc-3f67-40f1-9bcd-3cfa8cd1dd96 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:07.815708 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1ed4f-e106-496b-aebd-f157ce9a32e3 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:07.846707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:07.859312 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2768d-383f-4282-971f-385e3240a1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:07.860016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2768d-383f-4282-971f-385e3240a1ef None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:07.860308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2768d-383f-4282-971f-385e3240a1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:07.860583 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2768d-383f-4282-971f-385e3240a1ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:07.861383 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2768d-383f-4282-971f-385e3240a1ef None admin] RBAC: Adding query filter params (interface=public, service_id=c41d18f5ac51439aa14ef1a83861e7b2, region_id=RegionOne) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:07.861777 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2768d-383f-4282-971f-385e3240a1ef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:07.863299 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2768d-383f-4282-971f-385e3240a1ef None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:07.869315 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 450/910] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jul 21 08:57:07 2022] GET /identity/v3/endpoints?service_id=c41d18f5ac51439aa14ef1a83861e7b2&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:07.875270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2768d-383f-4282-971f-385e3240a1ef None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:07.942269 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51317bc-3f67-40f1-9bcd-3cfa8cd1dd96 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:07.982844 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b85faa-8f90-4ead-8b67-9e42990737af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:07.983785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b85faa-8f90-4ead-8b67-9e42990737af None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:07.984145 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b85faa-8f90-4ead-8b67-9e42990737af None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:07.984558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b85faa-8f90-4ead-8b67-9e42990737af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:07.985485 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b85faa-8f90-4ead-8b67-9e42990737af None admin] RBAC: Adding query filter params (role.id=1d3b684bd8714b5792ebd7695c02a9ab, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=257450db238d4efda783e30cbcd23d4f) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:07.985867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b85faa-8f90-4ead-8b67-9e42990737af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:07.987497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b85faa-8f90-4ead-8b67-9e42990737af None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:07.999952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 461/911] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:57:07 2022] GET /identity/v3/role_assignments?user.id=257450db238d4efda783e30cbcd23d4f&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=1d3b684bd8714b5792ebd7695c02a9ab => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:08.019901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2768d-383f-4282-971f-385e3240a1ef None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:08.061915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a29411-0d5d-4424-be93-0b1e43718afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:08.063140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a29411-0d5d-4424-be93-0b1e43718afa None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:08.063481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a29411-0d5d-4424-be93-0b1e43718afa None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:08.063710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a29411-0d5d-4424-be93-0b1e43718afa None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:08.064485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a29411-0d5d-4424-be93-0b1e43718afa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:08.065968 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a29411-0d5d-4424-be93-0b1e43718afa None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:08.073783 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 451/912] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jul 21 08:57:07 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:57:08.449439 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bbb76-ff48-4f0d-868c-778b20c5062e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:08.449795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bbb76-ff48-4f0d-868c-778b20c5062e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:08.449985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bbb76-ff48-4f0d-868c-778b20c5062e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:08.451611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 462/913] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:08.457542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bc943-9be9-4026-a60f-21c1ca886ff7 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:08.458022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bc943-9be9-4026-a60f-21c1ca886ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:08.458098 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bc943-9be9-4026-a60f-21c1ca886ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:08.475796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-255bc943-9be9-4026-a60f-21c1ca886ff7 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:08.477039 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-255bc943-9be9-4026-a60f-21c1ca886ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:08.529491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:08.548922 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 452/914] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:08.571489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bbb76-ff48-4f0d-868c-778b20c5062e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:08.574467 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:08.686769 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bbb76-ff48-4f0d-868c-778b20c5062e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:08.728576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12247b-fc35-4db3-818c-54fd1fddd89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:08.729651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12247b-fc35-4db3-818c-54fd1fddd89d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:08.730074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12247b-fc35-4db3-818c-54fd1fddd89d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:08.730558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12247b-fc35-4db3-818c-54fd1fddd89d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:08.731357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12247b-fc35-4db3-818c-54fd1fddd89d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:08.732964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12247b-fc35-4db3-818c-54fd1fddd89d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-5e12247b-fc35-4db3-818c-54fd1fddd89d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:08.746365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:08.747598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:08.748611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:08.748611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:08.748611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:08.748611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:08.748611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:08.748611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:08.748611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:08.748611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:08.748611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:08.748611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:08.748611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:08.748611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:08.748611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:57:08.748611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:08.751330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 463/915] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:57:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:08.757629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bc943-9be9-4026-a60f-21c1ca886ff7 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:08.863671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bc943-9be9-4026-a60f-21c1ca886ff7 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:08.899319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5ade8-f487-4e4e-8a7c-be42cfcff422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:08.900290 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5ade8-f487-4e4e-8a7c-be42cfcff422 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:08.900716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5ade8-f487-4e4e-8a7c-be42cfcff422 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:08.901129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5ade8-f487-4e4e-8a7c-be42cfcff422 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:08.901971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5ade8-f487-4e4e-8a7c-be42cfcff422 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:08.902417 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5ade8-f487-4e4e-8a7c-be42cfcff422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:08.903996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5ade8-f487-4e4e-8a7c-be42cfcff422 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:08.909868 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 453/916] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:57:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:08.915764 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12247b-fc35-4db3-818c-54fd1fddd89d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:08.991741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:09.029523 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12247b-fc35-4db3-818c-54fd1fddd89d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:09.065540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2c765-9a92-47c5-9470-45b16b1837d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:09.066208 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2c765-9a92-47c5-9470-45b16b1837d9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:09.066493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2c765-9a92-47c5-9470-45b16b1837d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:09.066765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2c765-9a92-47c5-9470-45b16b1837d9 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:09.067480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2c765-9a92-47c5-9470-45b16b1837d9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:09.067783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2c765-9a92-47c5-9470-45b16b1837d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:09.069241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2c765-9a92-47c5-9470-45b16b1837d9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:09.074947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 464/917] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jul 21 08:57:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:09.081266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5ade8-f487-4e4e-8a7c-be42cfcff422 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:09.133876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:09.189797 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5ade8-f487-4e4e-8a7c-be42cfcff422 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:09.227988 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e8d84-1b57-4765-b43a-f9768b3a1c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:09.229040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e8d84-1b57-4765-b43a-f9768b3a1c03 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:09.229535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e8d84-1b57-4765-b43a-f9768b3a1c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:09.230011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e8d84-1b57-4765-b43a-f9768b3a1c03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:09.231020 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e8d84-1b57-4765-b43a-f9768b3a1c03 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:09.232739 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e8d84-1b57-4765-b43a-f9768b3a1c03 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:09.259173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 454/918] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jul 21 08:57:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:57:09.436828 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f678ea4-23dc-4545-af1e-d7dd4689a286 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:09.437980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f678ea4-23dc-4545-af1e-d7dd4689a286 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:09.437980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f678ea4-23dc-4545-af1e-d7dd4689a286 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:09.439645 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 465/919] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:09.445281 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0584797-3dc5-4945-b515-fbb39d8f15d5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:09.445568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0584797-3dc5-4945-b515-fbb39d8f15d5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:09.445840 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0584797-3dc5-4945-b515-fbb39d8f15d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:09.464743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b0584797-3dc5-4945-b515-fbb39d8f15d5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:09.465929 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b0584797-3dc5-4945-b515-fbb39d8f15d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:09.541453 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 455/920] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:09.565715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f678ea4-23dc-4545-af1e-d7dd4689a286 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:09.678425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f678ea4-23dc-4545-af1e-d7dd4689a286 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:09.709555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:09.719965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604d833-c13c-4ac9-a5a4-5f1e91719d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:09.719965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604d833-c13c-4ac9-a5a4-5f1e91719d9c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:09.719965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604d833-c13c-4ac9-a5a4-5f1e91719d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:09.719965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604d833-c13c-4ac9-a5a4-5f1e91719d9c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:09.720361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604d833-c13c-4ac9-a5a4-5f1e91719d9c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:09.720361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604d833-c13c-4ac9-a5a4-5f1e91719d9c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-4604d833-c13c-4ac9-a5a4-5f1e91719d9c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:09.732918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:09.734129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:09.735150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:09.735150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:09.735150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:09.735150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:09.735150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:09.735150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:09.735150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:09.735150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:09.735150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:09.735150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:09.735150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:09.735150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:09.735150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:57:09.735150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:09.735150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 466/921] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jul 21 08:57:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:09.740606 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0584797-3dc5-4945-b515-fbb39d8f15d5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:09.773028 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:09.856583 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0584797-3dc5-4945-b515-fbb39d8f15d5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:09.893417 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c7db1-0d5a-4fa3-b296-9fd14e675691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:09.894080 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c7db1-0d5a-4fa3-b296-9fd14e675691 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:09.894359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c7db1-0d5a-4fa3-b296-9fd14e675691 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:09.894629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c7db1-0d5a-4fa3-b296-9fd14e675691 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:09.895332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c7db1-0d5a-4fa3-b296-9fd14e675691 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:09.895632 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c7db1-0d5a-4fa3-b296-9fd14e675691 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:09.897040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c7db1-0d5a-4fa3-b296-9fd14e675691 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:09.902498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 456/922] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jul 21 08:57:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:09.909447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604d833-c13c-4ac9-a5a4-5f1e91719d9c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:10.020252 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604d833-c13c-4ac9-a5a4-5f1e91719d9c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:10.056839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bfa10a-ef5c-47cd-947c-3f5c474d72e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:10.057524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfa10a-ef5c-47cd-947c-3f5c474d72e0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:10.057810 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfa10a-ef5c-47cd-947c-3f5c474d72e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:10.058084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bfa10a-ef5c-47cd-947c-3f5c474d72e0 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:10.058806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfa10a-ef5c-47cd-947c-3f5c474d72e0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:10.059174 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfa10a-ef5c-47cd-947c-3f5c474d72e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:10.060629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bfa10a-ef5c-47cd-947c-3f5c474d72e0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:10.066675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 467/923] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jul 21 08:57:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:10.072605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c7db1-0d5a-4fa3-b296-9fd14e675691 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:10.173767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:10.180675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c7db1-0d5a-4fa3-b296-9fd14e675691 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:10.182939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6e52a-545f-4b16-8cfb-e31085786f51 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:10.183228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6e52a-545f-4b16-8cfb-e31085786f51 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:10.183509 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6e52a-545f-4b16-8cfb-e31085786f51 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:10.186173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 468/924] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:10.190402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aca2ba-e632-4945-a425-96cc22ecb526 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:10.190698 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aca2ba-e632-4945-a425-96cc22ecb526 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:10.190981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aca2ba-e632-4945-a425-96cc22ecb526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:10.209371 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c7aca2ba-e632-4945-a425-96cc22ecb526 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:10.210705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c7aca2ba-e632-4945-a425-96cc22ecb526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:10.216340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ab954-ba1c-4579-9782-c766e43a41be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:10.216991 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ab954-ba1c-4579-9782-c766e43a41be None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:10.217271 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ab954-ba1c-4579-9782-c766e43a41be None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:10.217542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ab954-ba1c-4579-9782-c766e43a41be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:10.218880 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ab954-ba1c-4579-9782-c766e43a41be None admin] RBAC: Adding query filter params (interface=public, service_id=bbed50a5e9af4b8194d4ed09ccd43998, region_id=RegionOne) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:10.219172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ab954-ba1c-4579-9782-c766e43a41be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:10.220847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ab954-ba1c-4579-9782-c766e43a41be None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:10.226597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 457/925] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jul 21 08:57:10 2022] GET /identity/v3/endpoints?service_id=bbed50a5e9af4b8194d4ed09ccd43998&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:10.227547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f4c2a-1dc6-4b5a-b50b-9f555a2422bb None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:10.227828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f4c2a-1dc6-4b5a-b50b-9f555a2422bb None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:10.228128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f4c2a-1dc6-4b5a-b50b-9f555a2422bb None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:10.229471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 458/926] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:10.234515 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f4c2a-1dc6-4b5a-b50b-9f555a2422bb None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:10.272068 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:10.288409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 469/927] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:10.296078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f358e8-49b0-4c14-9af6-48405b251625 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:10.296078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f358e8-49b0-4c14-9af6-48405b251625 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:10.296078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f358e8-49b0-4c14-9af6-48405b251625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:10.307631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-93f358e8-49b0-4c14-9af6-48405b251625 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:10.308873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-93f358e8-49b0-4c14-9af6-48405b251625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:10.366219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f4c2a-1dc6-4b5a-b50b-9f555a2422bb None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:10.388623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 470/928] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:10.389766 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f358e8-49b0-4c14-9af6-48405b251625 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:10.412700 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c2745-2dae-4f82-b57b-2d74d05f9684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:10.413368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c2745-2dae-4f82-b57b-2d74d05f9684 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:10.413649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c2745-2dae-4f82-b57b-2d74d05f9684 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:10.413923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c2745-2dae-4f82-b57b-2d74d05f9684 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:10.414595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c2745-2dae-4f82-b57b-2d74d05f9684 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:10.416014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c2745-2dae-4f82-b57b-2d74d05f9684 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:10.423184 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 459/929] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jul 21 08:57:10 2022] GET /identity/v3/services => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:57:10.423988 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c2745-2dae-4f82-b57b-2d74d05f9684 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:10.450627 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:10.545060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f358e8-49b0-4c14-9af6-48405b251625 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:10.546404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c2745-2dae-4f82-b57b-2d74d05f9684 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:10.567130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:10.587748 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708dbda5-94c1-4458-adb6-1503c947137b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:10.588858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708dbda5-94c1-4458-adb6-1503c947137b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:10.589318 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf17f72-a30f-45cd-bb56-bb70a513e35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:10.589472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708dbda5-94c1-4458-adb6-1503c947137b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:10.589927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708dbda5-94c1-4458-adb6-1503c947137b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:10.590132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf17f72-a30f-45cd-bb56-bb70a513e35e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:10.590414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf17f72-a30f-45cd-bb56-bb70a513e35e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:10.590685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf17f72-a30f-45cd-bb56-bb70a513e35e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:10.592038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf17f72-a30f-45cd-bb56-bb70a513e35e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:10.593965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf17f72-a30f-45cd-bb56-bb70a513e35e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:10.604424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708dbda5-94c1-4458-adb6-1503c947137b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-fbf17f72-a30f-45cd-bb56-bb70a513e35e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:10.607507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:10.608544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:10.609580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:10.609580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:10.609580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:10.609580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:10.609580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:10.609580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:10.609580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:10.609580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:10.609580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:10.609580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:10.609580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:10.609580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:10.609580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:10.609580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:57:10.609580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:10.610497 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 460/930] 10.222.0.13 () {56 vars in 1132 bytes} [Thu Jul 21 08:57:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-708dbda5-94c1-4458-adb6-1503c947137b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:57:10.616242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:10.617310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:10.618151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:10.618151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:10.618151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:10.618151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:10.618151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:57:10.618151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:57:10.618151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:57:10.618151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:57:10.618151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:57:10.618151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:57:10.618151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:57:10.618151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:10.618783 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf17f72-a30f-45cd-bb56-bb70a513e35e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:10.621647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 471/931] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jul 21 08:57:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:10.628967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708dbda5-94c1-4458-adb6-1503c947137b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:10.746210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:10.757724 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708dbda5-94c1-4458-adb6-1503c947137b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:10.767587 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf17f72-a30f-45cd-bb56-bb70a513e35e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:10.799742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1ffdd-e2b8-45ea-b922-dd2926184f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:10.800616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1ffdd-e2b8-45ea-b922-dd2926184f70 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:10.801004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1ffdd-e2b8-45ea-b922-dd2926184f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:10.801668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1ffdd-e2b8-45ea-b922-dd2926184f70 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:10.802533 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1ffdd-e2b8-45ea-b922-dd2926184f70 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:10.802978 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1ffdd-e2b8-45ea-b922-dd2926184f70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:10.805202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1ffdd-e2b8-45ea-b922-dd2926184f70 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:10.821084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 472/932] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jul 21 08:57:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:10.821288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c832159-16b9-41ba-8370-d3346b50f38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:10.821288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c832159-16b9-41ba-8370-d3346b50f38b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:10.821288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c832159-16b9-41ba-8370-d3346b50f38b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:10.821523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c832159-16b9-41ba-8370-d3346b50f38b None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:10.822327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c832159-16b9-41ba-8370-d3346b50f38b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:10.822429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c832159-16b9-41ba-8370-d3346b50f38b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:10.823854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c832159-16b9-41ba-8370-d3346b50f38b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:10.824848 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1ffdd-e2b8-45ea-b922-dd2926184f70 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:10.836588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 461/933] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jul 21 08:57:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:10.837532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c832159-16b9-41ba-8370-d3346b50f38b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:10.880854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:10.946941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c832159-16b9-41ba-8370-d3346b50f38b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:10.951120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1ffdd-e2b8-45ea-b922-dd2926184f70 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:10.983051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773b048-9238-45e3-8f35-def05d88f1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:10.983647 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773b048-9238-45e3-8f35-def05d88f1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:10.984498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773b048-9238-45e3-8f35-def05d88f1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:10.984979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773b048-9238-45e3-8f35-def05d88f1d4 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:10.985873 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773b048-9238-45e3-8f35-def05d88f1d4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:10.986049 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773b048-9238-45e3-8f35-def05d88f1d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:10.987438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773b048-9238-45e3-8f35-def05d88f1d4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:10.991507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb2712-0c37-4742-ac0d-664c2408dc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:10.992609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb2712-0c37-4742-ac0d-664c2408dc00 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:10.993143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb2712-0c37-4742-ac0d-664c2408dc00 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:10.993610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb2712-0c37-4742-ac0d-664c2408dc00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:10.996054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 462/934] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jul 21 08:57:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:10.999107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773b048-9238-45e3-8f35-def05d88f1d4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:11.012075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb2712-0c37-4742-ac0d-664c2408dc00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:11.013896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb2712-0c37-4742-ac0d-664c2408dc00 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:11.018025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:11.035151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-e9bb2712-0c37-4742-ac0d-664c2408dc00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:11.035151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:11.035151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:11.035151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:11.035151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:11.035151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:11.035151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:11.035151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:11.035151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:11.035151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:11.035151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:11.035151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:11.035151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:11.035151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:11.035151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:11.036508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:11.037484 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:11.037484 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:11.037484 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:11.037484 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:11.041193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 473/935] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:11.045171 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb2712-0c37-4742-ac0d-664c2408dc00 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:11.112685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773b048-9238-45e3-8f35-def05d88f1d4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:11.149422 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15515acb-48a8-4061-aba8-138eab3c0715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:11.150433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:11.150669 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15515acb-48a8-4061-aba8-138eab3c0715 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:11.151151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15515acb-48a8-4061-aba8-138eab3c0715 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:11.151647 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15515acb-48a8-4061-aba8-138eab3c0715 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:11.152649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15515acb-48a8-4061-aba8-138eab3c0715 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:11.154296 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15515acb-48a8-4061-aba8-138eab3c0715 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:11.161952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb2712-0c37-4742-ac0d-664c2408dc00 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:11.188589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 463/936] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jul 21 08:57:10 2022] POST /identity/v3/endpoints => generated 347 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:57:11.215848 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd0bd2-2f12-464d-a0a6-e26454715952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:11.217058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd0bd2-2f12-464d-a0a6-e26454715952 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:11.217614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd0bd2-2f12-464d-a0a6-e26454715952 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:11.218102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd0bd2-2f12-464d-a0a6-e26454715952 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:11.220024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd0bd2-2f12-464d-a0a6-e26454715952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:11.220581 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd0bd2-2f12-464d-a0a6-e26454715952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:11.222375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd0bd2-2f12-464d-a0a6-e26454715952 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:11.237852 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 474/937] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:11.244268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15515acb-48a8-4061-aba8-138eab3c0715 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:11.331068 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:11.374138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15515acb-48a8-4061-aba8-138eab3c0715 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:11.416427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f66dd8-6a22-4843-8aa7-4a5539b78f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:11.417261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f66dd8-6a22-4843-8aa7-4a5539b78f74 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:11.417540 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f66dd8-6a22-4843-8aa7-4a5539b78f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:11.417815 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f66dd8-6a22-4843-8aa7-4a5539b78f74 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:11.427581 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f66dd8-6a22-4843-8aa7-4a5539b78f74 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:11.429138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f66dd8-6a22-4843-8aa7-4a5539b78f74 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-e2f66dd8-6a22-4843-8aa7-4a5539b78f74 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:57:11.446522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:11.447529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:11.448611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:11.448611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:11.448611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:57:11.448611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:57:11.448611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:57:11.448611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:57:11.448611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:57:11.448611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:57:11.448611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:57:11.448611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:11.452177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 464/938] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jul 21 08:57:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:11.455841 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd0bd2-2f12-464d-a0a6-e26454715952 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:11.480403 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1927f95-fbce-4dae-b489-ce2adabc3cdd None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:11.480744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1927f95-fbce-4dae-b489-ce2adabc3cdd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:11.480953 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1927f95-fbce-4dae-b489-ce2adabc3cdd None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:11.482358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 465/939] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:57:11.489311 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e722b3a-0f52-42fb-854e-dd3e68d1e31d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:11.489593 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e722b3a-0f52-42fb-854e-dd3e68d1e31d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:11.489788 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e722b3a-0f52-42fb-854e-dd3e68d1e31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:11.508226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1e722b3a-0f52-42fb-854e-dd3e68d1e31d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:11.509442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1e722b3a-0f52-42fb-854e-dd3e68d1e31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:11.588152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd0bd2-2f12-464d-a0a6-e26454715952 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:11.604159 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 466/940] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:11.622131 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e722b3a-0f52-42fb-854e-dd3e68d1e31d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:11.635209 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ea210-f640-4bcd-aab9-fa80f00af46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:11.635980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ea210-f640-4bcd-aab9-fa80f00af46d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:11.636270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ea210-f640-4bcd-aab9-fa80f00af46d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:11.636612 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ea210-f640-4bcd-aab9-fa80f00af46d None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:11.637468 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ea210-f640-4bcd-aab9-fa80f00af46d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:11.637781 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ea210-f640-4bcd-aab9-fa80f00af46d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:11.639235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ea210-f640-4bcd-aab9-fa80f00af46d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:11.661014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 475/941] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jul 21 08:57:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:11.667156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ea210-f640-4bcd-aab9-fa80f00af46d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:11.684698 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:11.699757 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:11.754046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e722b3a-0f52-42fb-854e-dd3e68d1e31d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:11.802963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbc4a0-3048-4f43-9a7d-b8339a6454a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:11.803755 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbc4a0-3048-4f43-9a7d-b8339a6454a0 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:11.804145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbc4a0-3048-4f43-9a7d-b8339a6454a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:11.804491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbc4a0-3048-4f43-9a7d-b8339a6454a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:11.804642 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ea210-f640-4bcd-aab9-fa80f00af46d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:11.805329 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbc4a0-3048-4f43-9a7d-b8339a6454a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:11.807832 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbc4a0-3048-4f43-9a7d-b8339a6454a0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:11.809438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: WARNING keystone.server.flask.common [None req-b6bbc4a0-3048-4f43-9a7d-b8339a6454a0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 08:57:11.832092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.sql.core [None req-b6bbc4a0-3048-4f43-9a7d-b8339a6454a0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 08:57:11.832092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 08:57:11.832092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [parameters: {'id': '6809473bf233407cbc4861988157f826', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 08:57:11.832092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80199) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 08:57:11.839726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b6bbc4a0-3048-4f43-9a7d-b8339a6454a0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6809473bf233407cbc4861988157f826.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6809473bf233407cbc4861988157f826. Jul 21 08:57:11.839726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:11.839726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 08:57:11.839726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 08:57:11.839726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 08:57:11.839726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 08:57:11.839726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 08:57:11.839726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 08:57:11.839726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 08:57:11.839726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application conn.query(q) Jul 21 08:57:11.839726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 08:57:11.839726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 08:57:11.839726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application result.read() Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 21 08:57:11.840566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application next(self.gen) Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application yield resource Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application next(self.gen) Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise self.value Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application session.commit() Jul 21 08:57:11.841891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self.session.flush() Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application compat.raise_( Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 08:57:11.843054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 08:57:11.844224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 08:57:11.844224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 08:57:11.844224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 08:57:11.844224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 08:57:11.844224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 08:57:11.844224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 08:57:11.844224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 08:57:11.844224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 08:57:11.844224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 08:57:11.844224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 08:57:11.844224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 08:57:11.844224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 08:57:11.844224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 08:57:11.844224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 08:57:11.844224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application conn.query(q) Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application result.read() Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 08:57:11.848474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 08:57:11.850243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 08:57:11.850243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 08:57:11.850243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 08:57:11.850243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 08:57:11.850243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 08:57:11.850243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 08:57:11.850243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 08:57:11.850243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 08:57:11.850243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [parameters: {'id': '6809473bf233407cbc4861988157f826', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 08:57:11.850243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 08:57:11.850243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:11.850243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:11.850243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:11.850243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:11.850243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:11.851652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:11.853375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:11.853375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 21 08:57:11.853375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 21 08:57:11.853375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:11.853375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:11.853375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 21 08:57:11.853375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 21 08:57:11.853375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6809473bf233407cbc4861988157f826. Jul 21 08:57:11.853375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:11.853375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 467/942] 10.222.0.13 () {60 vars in 1140 bytes} [Thu Jul 21 08:57:11 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:11.854127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d48e4e-1146-40f1-ad07-4b5efddb3866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:11.854127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d48e4e-1146-40f1-ad07-4b5efddb3866 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:11.854127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d48e4e-1146-40f1-ad07-4b5efddb3866 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:11.854127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d48e4e-1146-40f1-ad07-4b5efddb3866 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:11.856224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbc4a0-3048-4f43-9a7d-b8339a6454a0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:11.864089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d48e4e-1146-40f1-ad07-4b5efddb3866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:11.864089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d48e4e-1146-40f1-ad07-4b5efddb3866 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:11.885212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-55d48e4e-1146-40f1-ad07-4b5efddb3866 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:11.885212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:11.885212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:11.885212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:11.885212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:11.885212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:11.885212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:11.885212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:11.885212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:11.885212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:11.885212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:11.885212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:11.885212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:11.885212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:11.885212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:11.886778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:11.888775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:11.888775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:11.888775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:11.888775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:11.888775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 476/943] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:11.897694 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d48e4e-1146-40f1-ad07-4b5efddb3866 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:11.964744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbc4a0-3048-4f43-9a7d-b8339a6454a0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:11.967176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:11.983299 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:12.004493 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b68b62-e06e-4e4c-9ce1-5e619d73ef03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:12.005363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b68b62-e06e-4e4c-9ce1-5e619d73ef03 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:12.005748 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b68b62-e06e-4e4c-9ce1-5e619d73ef03 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:12.006143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b68b62-e06e-4e4c-9ce1-5e619d73ef03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:12.013780 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b68b62-e06e-4e4c-9ce1-5e619d73ef03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:12.016468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b68b62-e06e-4e4c-9ce1-5e619d73ef03 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:12.021917 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d48e4e-1146-40f1-ad07-4b5efddb3866 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:12.032639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-f6b68b62-e06e-4e4c-9ce1-5e619d73ef03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:12.032639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:12.032639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:12.032639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:12.032639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:12.032639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:12.032639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:12.032639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:12.032639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:12.032639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:12.032639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:12.032639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:12.032639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:57:12.032639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:57:12.032639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:12.033604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:12.034754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:57:12.034754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:57:12.034754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:57:12.034754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:57:12.034754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:57:12.034754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:57:12.034754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:12.034754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:12.034754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 468/944] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:57:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:12.039874 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b68b62-e06e-4e4c-9ce1-5e619d73ef03 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:12.065287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95af746-23c7-4d7b-8580-f857f981d1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:12.066744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95af746-23c7-4d7b-8580-f857f981d1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:12.067201 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95af746-23c7-4d7b-8580-f857f981d1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:12.067698 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95af746-23c7-4d7b-8580-f857f981d1a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:12.068745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95af746-23c7-4d7b-8580-f857f981d1a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:12.069230 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95af746-23c7-4d7b-8580-f857f981d1a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:12.070952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95af746-23c7-4d7b-8580-f857f981d1a6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:12.090390 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 477/945] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:12.092086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95af746-23c7-4d7b-8580-f857f981d1a6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:12.152177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b68b62-e06e-4e4c-9ce1-5e619d73ef03 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:12.189255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707dc85-fc2f-443d-af9a-10b3117b3dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:12.190182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707dc85-fc2f-443d-af9a-10b3117b3dcf None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:12.190543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707dc85-fc2f-443d-af9a-10b3117b3dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:12.190911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707dc85-fc2f-443d-af9a-10b3117b3dcf None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:12.191747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707dc85-fc2f-443d-af9a-10b3117b3dcf None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:12.192151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707dc85-fc2f-443d-af9a-10b3117b3dcf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:12.193681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707dc85-fc2f-443d-af9a-10b3117b3dcf None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:12.207247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95af746-23c7-4d7b-8580-f857f981d1a6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:12.220245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 469/946] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jul 21 08:57:12 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:12.227159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:12.251098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee3a01-2ca3-4f46-89f6-431ab497a517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:12.252322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee3a01-2ca3-4f46-89f6-431ab497a517 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:12.252781 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee3a01-2ca3-4f46-89f6-431ab497a517 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:12.253224 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee3a01-2ca3-4f46-89f6-431ab497a517 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:12.261714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee3a01-2ca3-4f46-89f6-431ab497a517 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:12.263475 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee3a01-2ca3-4f46-89f6-431ab497a517 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:12.280146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-66ee3a01-2ca3-4f46-89f6-431ab497a517 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:12.280146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:12.280146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:12.280146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:12.280146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:12.280146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:12.280146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:12.280146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:12.280146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:12.280146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:12.280146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:12.280146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:12.280146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:57:12.280146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:57:12.280146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:12.281153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:12.282061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:57:12.282061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:57:12.282061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:57:12.282061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:57:12.282061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:57:12.282061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:57:12.282061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:12.282061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:12.289042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 478/947] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:57:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:12.290889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707dc85-fc2f-443d-af9a-10b3117b3dcf None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:12.333742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:12.415099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707dc85-fc2f-443d-af9a-10b3117b3dcf None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:12.454215 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff06246-203a-445b-a562-43ab0042984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:12.454890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff06246-203a-445b-a562-43ab0042984f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:12.455179 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff06246-203a-445b-a562-43ab0042984f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:12.455457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff06246-203a-445b-a562-43ab0042984f None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:12.457153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff06246-203a-445b-a562-43ab0042984f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:12.457464 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff06246-203a-445b-a562-43ab0042984f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:12.458997 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff06246-203a-445b-a562-43ab0042984f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:12.473855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 470/948] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:57:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:12.480548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee3a01-2ca3-4f46-89f6-431ab497a517 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:12.584917 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603071aa-f7e3-41c0-a3de-39e28326c63e None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:12.585219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603071aa-f7e3-41c0-a3de-39e28326c63e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:12.585411 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603071aa-f7e3-41c0-a3de-39e28326c63e None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:12.587043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 471/949] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:12.592613 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b2da3-9f5d-472d-9b72-8e1ca9b67914 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:12.592898 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b2da3-9f5d-472d-9b72-8e1ca9b67914 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:12.593187 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b2da3-9f5d-472d-9b72-8e1ca9b67914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:12.593679 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee3a01-2ca3-4f46-89f6-431ab497a517 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:12.615476 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-432b2da3-9f5d-472d-9b72-8e1ca9b67914 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:12.617177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-432b2da3-9f5d-472d-9b72-8e1ca9b67914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:12.635937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f82f6-67eb-4f20-a4fd-e02d51a0f235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:12.635937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f82f6-67eb-4f20-a4fd-e02d51a0f235 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:12.635937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f82f6-67eb-4f20-a4fd-e02d51a0f235 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:12.635937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f82f6-67eb-4f20-a4fd-e02d51a0f235 None admin] PATH_INFO: `/v3/projects/7fe0f489cbce43859f050c3270e00709/users/257450db238d4efda783e30cbcd23d4f/roles/1d3b684bd8714b5792ebd7695c02a9ab` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:12.636354 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f82f6-67eb-4f20-a4fd-e02d51a0f235 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fe0f489cbce43859f050c3270e00709, user_id=257450db238d4efda783e30cbcd23d4f, role_id=1d3b684bd8714b5792ebd7695c02a9ab)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:12.637664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f82f6-67eb-4f20-a4fd-e02d51a0f235 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:12.660036 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 479/950] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:57:12 2022] PUT /identity/v3/projects/7fe0f489cbce43859f050c3270e00709/users/257450db238d4efda783e30cbcd23d4f/roles/1d3b684bd8714b5792ebd7695c02a9ab => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:57:12.708560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 472/951] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:12.732959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f82f6-67eb-4f20-a4fd-e02d51a0f235 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:12.778665 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:12.855950 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f82f6-67eb-4f20-a4fd-e02d51a0f235 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:12.898169 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f06b7c-43e4-434c-ac75-f9ee12dc3685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:12.899362 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f06b7c-43e4-434c-ac75-f9ee12dc3685 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:12.899849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f06b7c-43e4-434c-ac75-f9ee12dc3685 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:12.900354 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f06b7c-43e4-434c-ac75-f9ee12dc3685 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:12.901223 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f06b7c-43e4-434c-ac75-f9ee12dc3685 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:12.902615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f06b7c-43e4-434c-ac75-f9ee12dc3685 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-78f06b7c-43e4-434c-ac75-f9ee12dc3685 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:12.915203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:12.916261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:12.917932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:12.917932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:12.917932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:12.917932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:12.917932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:12.917932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:12.917932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:12.917932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:12.917932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:12.917932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:12.917932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:12.917932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:12.917932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:57:12.917932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:12.917932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 480/952] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jul 21 08:57:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:12.923769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b2da3-9f5d-472d-9b72-8e1ca9b67914 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:12.954751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:13.038211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b2da3-9f5d-472d-9b72-8e1ca9b67914 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:13.078386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2fdd2-567d-4918-a133-bca9383fce6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:13.079991 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2fdd2-567d-4918-a133-bca9383fce6f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:13.080801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2fdd2-567d-4918-a133-bca9383fce6f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:13.081562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2fdd2-567d-4918-a133-bca9383fce6f None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:13.083138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2fdd2-567d-4918-a133-bca9383fce6f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:13.083941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2fdd2-567d-4918-a133-bca9383fce6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:13.085593 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2fdd2-567d-4918-a133-bca9383fce6f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:13.091628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 473/953] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jul 21 08:57:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:13.097425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f06b7c-43e4-434c-ac75-f9ee12dc3685 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:13.210507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f06b7c-43e4-434c-ac75-f9ee12dc3685 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:13.212474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:13.252499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb9442-123d-4740-9a1f-f717a8e41aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:13.253176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb9442-123d-4740-9a1f-f717a8e41aad None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:13.253461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb9442-123d-4740-9a1f-f717a8e41aad None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:13.253738 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb9442-123d-4740-9a1f-f717a8e41aad None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:13.254452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb9442-123d-4740-9a1f-f717a8e41aad None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:13.254762 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb9442-123d-4740-9a1f-f717a8e41aad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:13.256170 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb9442-123d-4740-9a1f-f717a8e41aad None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:13.262016 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 481/954] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jul 21 08:57:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:13.267465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2fdd2-567d-4918-a133-bca9383fce6f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:13.365244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:13.376370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2fdd2-567d-4918-a133-bca9383fce6f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:13.413199 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b5f1e-1e4e-4795-939c-d08838b4377c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:13.414307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b5f1e-1e4e-4795-939c-d08838b4377c None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:13.414799 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b5f1e-1e4e-4795-939c-d08838b4377c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:13.415276 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b5f1e-1e4e-4795-939c-d08838b4377c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:13.416247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b5f1e-1e4e-4795-939c-d08838b4377c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:13.417839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b5f1e-1e4e-4795-939c-d08838b4377c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:13.441868 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 474/955] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jul 21 08:57:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:57:13.443396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f88d1-6a0d-423e-a0db-4318f8e3bbc2 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:13.443694 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f88d1-6a0d-423e-a0db-4318f8e3bbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:13.444007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f88d1-6a0d-423e-a0db-4318f8e3bbc2 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:13.445675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 482/956] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:13.451786 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a863b87-9085-441c-b456-18cf53102709 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:13.452529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a863b87-9085-441c-b456-18cf53102709 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:13.453115 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a863b87-9085-441c-b456-18cf53102709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:13.473985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-9a863b87-9085-441c-b456-18cf53102709 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:13.475591 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9a863b87-9085-441c-b456-18cf53102709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:13.572630 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 475/957] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:13.609895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f88d1-6a0d-423e-a0db-4318f8e3bbc2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:13.744924 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f88d1-6a0d-423e-a0db-4318f8e3bbc2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:13.797769 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35679b-87a6-405d-93e9-778e66fa1dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:13.798956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35679b-87a6-405d-93e9-778e66fa1dab None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:13.799453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35679b-87a6-405d-93e9-778e66fa1dab None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:13.799986 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35679b-87a6-405d-93e9-778e66fa1dab None admin] PATH_INFO: `/v3/services/compute` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:13.800878 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35679b-87a6-405d-93e9-778e66fa1dab None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:13.802511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35679b-87a6-405d-93e9-778e66fa1dab None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-5d35679b-87a6-405d-93e9-778e66fa1dab None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:13.815321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:13.816394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:13.817322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:13.817322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:13.817322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:13.817322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:13.817322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:13.817322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:13.817322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:13.817322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:13.817322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:13.817322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:13.817322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:13.817322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:13.817322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:57:13.817322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:13.820012 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 483/958] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:57:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:13.830056 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a863b87-9085-441c-b456-18cf53102709 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:13.930792 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:13.979045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a863b87-9085-441c-b456-18cf53102709 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:14.077651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead589d5-6573-4d55-9ac5-3f158a0bdf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:14.079170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead589d5-6573-4d55-9ac5-3f158a0bdf20 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:14.079641 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead589d5-6573-4d55-9ac5-3f158a0bdf20 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:14.080146 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead589d5-6573-4d55-9ac5-3f158a0bdf20 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:14.081046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead589d5-6573-4d55-9ac5-3f158a0bdf20 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:14.081534 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead589d5-6573-4d55-9ac5-3f158a0bdf20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:14.083464 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead589d5-6573-4d55-9ac5-3f158a0bdf20 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:14.094447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 476/959] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jul 21 08:57:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:14.104828 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35679b-87a6-405d-93e9-778e66fa1dab None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:14.154838 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:14.232273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35679b-87a6-405d-93e9-778e66fa1dab None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:14.268512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9edb6b-b52b-4bbb-9888-69366ce557bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:14.269421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9edb6b-b52b-4bbb-9888-69366ce557bf None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:14.269851 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9edb6b-b52b-4bbb-9888-69366ce557bf None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:14.270297 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9edb6b-b52b-4bbb-9888-69366ce557bf None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:14.271143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9edb6b-b52b-4bbb-9888-69366ce557bf None admin] RBAC: Adding query filter params (type=compute) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:14.271599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9edb6b-b52b-4bbb-9888-69366ce557bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:14.273232 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9edb6b-b52b-4bbb-9888-69366ce557bf None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:14.278478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 484/960] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jul 21 08:57:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:14.569648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf5255-9738-4b0e-92ea-e1f25faf5c27 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:14.569979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf5255-9738-4b0e-92ea-e1f25faf5c27 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:14.570206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf5255-9738-4b0e-92ea-e1f25faf5c27 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:14.571959 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 477/961] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:14.577272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc513c2e-746a-425e-ae0d-8014d04ac6d2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:14.577566 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc513c2e-746a-425e-ae0d-8014d04ac6d2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:14.578365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc513c2e-746a-425e-ae0d-8014d04ac6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:14.596104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-cc513c2e-746a-425e-ae0d-8014d04ac6d2 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:14.597294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cc513c2e-746a-425e-ae0d-8014d04ac6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:14.673776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 485/962] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:14.698887 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf5255-9738-4b0e-92ea-e1f25faf5c27 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:14.825508 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf5255-9738-4b0e-92ea-e1f25faf5c27 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:14.868947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba510ea-a2b9-401b-90cb-c29144864b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:14.870457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba510ea-a2b9-401b-90cb-c29144864b9a None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:14.870978 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba510ea-a2b9-401b-90cb-c29144864b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:14.871456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba510ea-a2b9-401b-90cb-c29144864b9a None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:14.872476 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba510ea-a2b9-401b-90cb-c29144864b9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:14.874227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba510ea-a2b9-401b-90cb-c29144864b9a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:14.884764 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:14.890263 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.sql.core [None req-2ba510ea-a2b9-401b-90cb-c29144864b9a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 08:57:14.890263 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 08:57:14.890263 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [parameters: {'user_id': '70e5644cdb034c01b1ebaf8e44ef1f4e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 08:57:14.890263 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80199) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 08:57:14.899355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-2ba510ea-a2b9-401b-90cb-c29144864b9a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 08:57:14.899355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:14.899355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 08:57:14.899355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 08:57:14.899355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 08:57:14.899355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 08:57:14.899355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 08:57:14.899355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 08:57:14.899355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 08:57:14.899355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application conn.query(q) Jul 21 08:57:14.899355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 08:57:14.899355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 08:57:14.899355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 08:57:14.899355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application result.read() Jul 21 08:57:14.899355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 08:57:14.900358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application next(self.gen) Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application yield resource Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application next(self.gen) Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise self.value Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application session.commit() Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 08:57:14.901337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self.session.flush() Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application compat.raise_( Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 08:57:14.902227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 08:57:14.903138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application conn.query(q) Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application result.read() Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 08:57:14.904945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 08:57:14.906999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 08:57:14.906999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 08:57:14.906999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 08:57:14.906999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 08:57:14.906999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 08:57:14.906999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 08:57:14.906999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [parameters: {'user_id': '70e5644cdb034c01b1ebaf8e44ef1f4e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 08:57:14.906999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 08:57:14.906999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:14.906999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:14.906999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:14.906999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:14.906999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:14.906999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:14.906999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 21 08:57:14.908974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 21 08:57:14.912342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 21 08:57:14.912342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 21 08:57:14.912342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 08:57:14.912342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 08:57:14.912342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 08:57:14.912342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:14.912342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 478/963] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Jul 21 08:57:14 2022] POST /identity/v3/users => generated 160 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:14.912760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc513c2e-746a-425e-ae0d-8014d04ac6d2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:14.936438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:14.979362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1e976-e69c-4f7c-90a8-9b489fb1d11d None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:14.980111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1e976-e69c-4f7c-90a8-9b489fb1d11d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:14.980622 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1e976-e69c-4f7c-90a8-9b489fb1d11d None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:14.985800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 479/964] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:57:14.993658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6741342-b113-430d-94d5-ea7976563eb1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:14.994277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6741342-b113-430d-94d5-ea7976563eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:14.994758 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6741342-b113-430d-94d5-ea7976563eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:15.014331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c6741342-b113-430d-94d5-ea7976563eb1 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:15.017246 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c6741342-b113-430d-94d5-ea7976563eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:15.023065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc513c2e-746a-425e-ae0d-8014d04ac6d2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:15.060562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bef90-4821-4623-8030-69003f29e88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:15.061367 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bef90-4821-4623-8030-69003f29e88e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:15.061646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bef90-4821-4623-8030-69003f29e88e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:15.061917 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bef90-4821-4623-8030-69003f29e88e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:15.070906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bef90-4821-4623-8030-69003f29e88e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:15.072366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bef90-4821-4623-8030-69003f29e88e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-d64bef90-4821-4623-8030-69003f29e88e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:57:15.088281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:15.089229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:15.090830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:15.090830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:15.090830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:57:15.090830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:57:15.090830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:57:15.090830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:57:15.090830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:57:15.090830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:57:15.090830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:57:15.090830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:15.090830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 486/965] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:57:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:15.096146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bef90-4821-4623-8030-69003f29e88e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:15.108020 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 480/966] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:15.126650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6741342-b113-430d-94d5-ea7976563eb1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:15.188675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:15.208208 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bef90-4821-4623-8030-69003f29e88e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:15.233239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6741342-b113-430d-94d5-ea7976563eb1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:15.246606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d990258-0a1f-408c-81d3-0a813aa606e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:15.247361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d990258-0a1f-408c-81d3-0a813aa606e9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:15.247567 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d990258-0a1f-408c-81d3-0a813aa606e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:15.247823 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d990258-0a1f-408c-81d3-0a813aa606e9 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:15.248663 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d990258-0a1f-408c-81d3-0a813aa606e9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:15.248964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d990258-0a1f-408c-81d3-0a813aa606e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:15.251256 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d990258-0a1f-408c-81d3-0a813aa606e9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:15.265323 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:15.274162 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904705ec-a45f-4e5c-a3e3-50b7e0c8f123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:15.275337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904705ec-a45f-4e5c-a3e3-50b7e0c8f123 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:15.275844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904705ec-a45f-4e5c-a3e3-50b7e0c8f123 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:15.276352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904705ec-a45f-4e5c-a3e3-50b7e0c8f123 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:15.289173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904705ec-a45f-4e5c-a3e3-50b7e0c8f123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-904705ec-a45f-4e5c-a3e3-50b7e0c8f123 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:57:15.299291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:15.300741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:15.301617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:15.301617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:15.301617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:15.301617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:15.301617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:57:15.301617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:57:15.301617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:57:15.301617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:57:15.301617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:57:15.301617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:57:15.301617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:57:15.301617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:15.302306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 487/967] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jul 21 08:57:15 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:15.309597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 481/968] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jul 21 08:57:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:15.311438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d990258-0a1f-408c-81d3-0a813aa606e9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:15.428089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d990258-0a1f-408c-81d3-0a813aa606e9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:15.466635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb2dfa-5c4b-4f14-9ecd-26db55b5eb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:15.467333 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb2dfa-5c4b-4f14-9ecd-26db55b5eb1f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:15.467618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb2dfa-5c4b-4f14-9ecd-26db55b5eb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:15.467896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb2dfa-5c4b-4f14-9ecd-26db55b5eb1f None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:15.469523 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb2dfa-5c4b-4f14-9ecd-26db55b5eb1f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:15.470090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb2dfa-5c4b-4f14-9ecd-26db55b5eb1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:15.471322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb2dfa-5c4b-4f14-9ecd-26db55b5eb1f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:15.485701 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 488/969] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jul 21 08:57:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:15.488477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904705ec-a45f-4e5c-a3e3-50b7e0c8f123 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:15.606564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904705ec-a45f-4e5c-a3e3-50b7e0c8f123 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:15.643600 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49626f-ef87-4f00-a1fa-00d26e56e8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:15.644462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49626f-ef87-4f00-a1fa-00d26e56e8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:15.644749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49626f-ef87-4f00-a1fa-00d26e56e8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:15.645027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49626f-ef87-4f00-a1fa-00d26e56e8b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:15.656744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49626f-ef87-4f00-a1fa-00d26e56e8b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:15.658499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49626f-ef87-4f00-a1fa-00d26e56e8b9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:15.676752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-7e49626f-ef87-4f00-a1fa-00d26e56e8b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:15.676752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:15.676752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:15.676752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:15.676752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:15.676752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:15.676752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:15.676752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:15.676752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:15.676752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:15.676752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:15.676752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:15.676752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:15.676752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:15.676752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:15.677756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:15.678707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:15.678707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:15.678707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:15.678707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:15.679483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 482/970] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:15.684266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb2dfa-5c4b-4f14-9ecd-26db55b5eb1f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:15.688480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:15.792892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb2dfa-5c4b-4f14-9ecd-26db55b5eb1f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:15.827451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c854d-7888-4658-ba8f-806e6e139739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:15.828385 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c854d-7888-4658-ba8f-806e6e139739 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:15.828927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c854d-7888-4658-ba8f-806e6e139739 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:15.829302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c854d-7888-4658-ba8f-806e6e139739 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:15.830120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c854d-7888-4658-ba8f-806e6e139739 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:15.830720 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c854d-7888-4658-ba8f-806e6e139739 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:15.832106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c854d-7888-4658-ba8f-806e6e139739 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:15.832253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e73ec0-1f4b-4212-8287-a66dd0056c55 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:15.832547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e73ec0-1f4b-4212-8287-a66dd0056c55 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:15.832826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e73ec0-1f4b-4212-8287-a66dd0056c55 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:15.834365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 483/971] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:15.839101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9719f4-8197-4810-a102-fdae53cdbe18 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:15.839394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9719f4-8197-4810-a102-fdae53cdbe18 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:15.839680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9719f4-8197-4810-a102-fdae53cdbe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:15.852288 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 489/972] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:15.853830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c854d-7888-4658-ba8f-806e6e139739 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:15.860416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1b9719f4-8197-4810-a102-fdae53cdbe18 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:15.861805 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1b9719f4-8197-4810-a102-fdae53cdbe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:15.910496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:15.919463 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:15.950144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 484/973] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:15.965645 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c854d-7888-4658-ba8f-806e6e139739 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:15.973590 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9719f4-8197-4810-a102-fdae53cdbe18 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:16.006680 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a608f-3efe-43b6-825e-ae014451a504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:16.007455 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a608f-3efe-43b6-825e-ae014451a504 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:16.007739 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a608f-3efe-43b6-825e-ae014451a504 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:16.008042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a608f-3efe-43b6-825e-ae014451a504 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:16.016637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a608f-3efe-43b6-825e-ae014451a504 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:16.018779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a608f-3efe-43b6-825e-ae014451a504 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-4b3a608f-3efe-43b6-825e-ae014451a504 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:57:16.038044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:16.039225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:16.045235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:16.045235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:16.045235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:57:16.045235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:57:16.045235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:57:16.045235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:57:16.045235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:57:16.045235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:57:16.045235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:57:16.045235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:16.045235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 490/974] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jul 21 08:57:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:16.045235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a608f-3efe-43b6-825e-ae014451a504 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:16.106896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9719f4-8197-4810-a102-fdae53cdbe18 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:16.149184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a608f-3efe-43b6-825e-ae014451a504 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:16.151008 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7783d-c6c6-4bdd-bb38-09576920ec0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:16.152135 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7783d-c6c6-4bdd-bb38-09576920ec0e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:16.152135 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7783d-c6c6-4bdd-bb38-09576920ec0e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:16.152397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7783d-c6c6-4bdd-bb38-09576920ec0e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:16.153030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b7783d-c6c6-4bdd-bb38-09576920ec0e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:16.154410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b7783d-c6c6-4bdd-bb38-09576920ec0e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-a6b7783d-c6c6-4bdd-bb38-09576920ec0e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:16.167047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:16.168090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:16.168947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:16.168947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:16.168947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:16.168947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:16.168947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:16.168947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:16.168947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:16.168947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:16.168947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:16.168947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:16.168947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:16.168947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:16.168947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:57:16.168947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:16.168947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 485/975] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jul 21 08:57:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:16.175114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7783d-c6c6-4bdd-bb38-09576920ec0e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:16.178033 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:16.191433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2d6e9-bdbf-45ea-95c6-a1287222e66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:16.192219 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2d6e9-bdbf-45ea-95c6-a1287222e66a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:16.192401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2d6e9-bdbf-45ea-95c6-a1287222e66a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:16.192667 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2d6e9-bdbf-45ea-95c6-a1287222e66a None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:16.193500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2d6e9-bdbf-45ea-95c6-a1287222e66a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:16.193803 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2d6e9-bdbf-45ea-95c6-a1287222e66a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:16.195341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a2d6e9-bdbf-45ea-95c6-a1287222e66a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:16.211839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:16.216959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 491/976] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jul 21 08:57:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:16.222638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2d6e9-bdbf-45ea-95c6-a1287222e66a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:16.286059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7783d-c6c6-4bdd-bb38-09576920ec0e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:16.328792 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6c1fc-6489-496d-89e0-197e2e8d6a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:16.329465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6c1fc-6489-496d-89e0-197e2e8d6a3d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:16.329750 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6c1fc-6489-496d-89e0-197e2e8d6a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:16.330029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6c1fc-6489-496d-89e0-197e2e8d6a3d None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:16.330743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6c1fc-6489-496d-89e0-197e2e8d6a3d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:16.330838 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a2d6e9-bdbf-45ea-95c6-a1287222e66a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:16.331053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6c1fc-6489-496d-89e0-197e2e8d6a3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:16.332509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6c1fc-6489-496d-89e0-197e2e8d6a3d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:16.342538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 486/977] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jul 21 08:57:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:16.343599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6c1fc-6489-496d-89e0-197e2e8d6a3d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:16.373770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96888bf8-22b1-4451-af71-e5c259e44f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:16.374595 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96888bf8-22b1-4451-af71-e5c259e44f4a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:16.375459 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96888bf8-22b1-4451-af71-e5c259e44f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:16.375747 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96888bf8-22b1-4451-af71-e5c259e44f4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:16.436160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:16.457035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:16.459452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96888bf8-22b1-4451-af71-e5c259e44f4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:16.461165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96888bf8-22b1-4451-af71-e5c259e44f4a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:16.461921 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6c1fc-6489-496d-89e0-197e2e8d6a3d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:16.498445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c0210-9e29-453c-a06b-1a28df1b028c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:16.499111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c0210-9e29-453c-a06b-1a28df1b028c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:16.499396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c0210-9e29-453c-a06b-1a28df1b028c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:16.499720 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c0210-9e29-453c-a06b-1a28df1b028c None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:16.500952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c0210-9e29-453c-a06b-1a28df1b028c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:16.501261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c0210-9e29-453c-a06b-1a28df1b028c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:16.502655 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c0210-9e29-453c-a06b-1a28df1b028c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:16.507825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 487/978] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jul 21 08:57:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:16.509137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f7198-b0bd-45b3-88e9-34eb1e4f1767 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:16.509430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f7198-b0bd-45b3-88e9-34eb1e4f1767 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:16.509710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f7198-b0bd-45b3-88e9-34eb1e4f1767 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:16.511092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 488/979] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:16.515718 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f7198-b0bd-45b3-88e9-34eb1e4f1767 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:16.561764 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 492/980] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:57:16 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 08:57:16.562998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6df12-6435-4e49-be40-de070e3eaadc None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:16.563275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6df12-6435-4e49-be40-de070e3eaadc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:16.563493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6df12-6435-4e49-be40-de070e3eaadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:16.579541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-83c6df12-6435-4e49-be40-de070e3eaadc None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:16.580815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-83c6df12-6435-4e49-be40-de070e3eaadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:16.618271 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f7198-b0bd-45b3-88e9-34eb1e4f1767 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:16.650632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 493/981] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:16.651729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6df12-6435-4e49-be40-de070e3eaadc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:16.666054 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:16.669184 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569ecb5-3c94-4f85-8a91-fae09b0cbf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:16.669925 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569ecb5-3c94-4f85-8a91-fae09b0cbf26 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:16.670125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569ecb5-3c94-4f85-8a91-fae09b0cbf26 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:16.670393 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569ecb5-3c94-4f85-8a91-fae09b0cbf26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:16.671469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569ecb5-3c94-4f85-8a91-fae09b0cbf26 None admin] RBAC: Adding query filter params (interface=admin, service_id=bbed50a5e9af4b8194d4ed09ccd43998, region_id=RegionOne) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:16.672617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569ecb5-3c94-4f85-8a91-fae09b0cbf26 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:16.674016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569ecb5-3c94-4f85-8a91-fae09b0cbf26 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:16.679926 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 489/982] 10.222.0.13 () {56 vars in 1249 bytes} [Thu Jul 21 08:57:16 2022] GET /identity/v3/endpoints?service_id=bbed50a5e9af4b8194d4ed09ccd43998&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:16.680736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569ecb5-3c94-4f85-8a91-fae09b0cbf26 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:16.707952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:16.754428 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6df12-6435-4e49-be40-de070e3eaadc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:16.786661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569ecb5-3c94-4f85-8a91-fae09b0cbf26 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:16.788856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf69a5-d0e1-4ad4-bd3e-91d9cc88ec62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:16.789646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf69a5-d0e1-4ad4-bd3e-91d9cc88ec62 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:16.789930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf69a5-d0e1-4ad4-bd3e-91d9cc88ec62 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:16.790204 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf69a5-d0e1-4ad4-bd3e-91d9cc88ec62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:16.801001 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf69a5-d0e1-4ad4-bd3e-91d9cc88ec62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:16.802499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cf69a5-d0e1-4ad4-bd3e-91d9cc88ec62 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:16.821325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-b3cf69a5-d0e1-4ad4-bd3e-91d9cc88ec62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:16.821325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:16.821325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:16.821325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:16.821325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:16.821325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:16.821325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:16.821325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:16.821325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:16.821325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:16.821325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:16.821325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:16.821325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:16.821325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:16.821325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:16.822280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:16.823176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:16.823176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:16.823176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:16.823176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:16.823404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60259e-f6c4-4e85-9dca-833133203392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:16.823505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 494/983] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:16.824107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60259e-f6c4-4e85-9dca-833133203392 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:16.824107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60259e-f6c4-4e85-9dca-833133203392 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:16.824107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60259e-f6c4-4e85-9dca-833133203392 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:16.824348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf69a5-d0e1-4ad4-bd3e-91d9cc88ec62 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:16.824512 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60259e-f6c4-4e85-9dca-833133203392 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:16.825393 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60259e-f6c4-4e85-9dca-833133203392 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:16.840122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 490/984] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jul 21 08:57:16 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:57:16.840995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60259e-f6c4-4e85-9dca-833133203392 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:16.908729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:16.949079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:16.955199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf69a5-d0e1-4ad4-bd3e-91d9cc88ec62 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:16.979756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60259e-f6c4-4e85-9dca-833133203392 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:16.998941 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3102466-f09e-4dd6-bf3b-c38d98283741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:16.998941 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3102466-f09e-4dd6-bf3b-c38d98283741 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:16.999375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3102466-f09e-4dd6-bf3b-c38d98283741 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:16.999536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3102466-f09e-4dd6-bf3b-c38d98283741 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:17.000315 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3102466-f09e-4dd6-bf3b-c38d98283741 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:17.001708 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3102466-f09e-4dd6-bf3b-c38d98283741 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:17.009690 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 495/985] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jul 21 08:57:16 2022] GET /identity/v3/services => generated 1727 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:57:17.021882 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878972b5-c55c-4372-aade-4c21ba8bf6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:17.022580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878972b5-c55c-4372-aade-4c21ba8bf6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:17.022863 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878972b5-c55c-4372-aade-4c21ba8bf6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:17.023137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878972b5-c55c-4372-aade-4c21ba8bf6e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:17.023886 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878972b5-c55c-4372-aade-4c21ba8bf6e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:17.024285 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878972b5-c55c-4372-aade-4c21ba8bf6e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:17.025743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878972b5-c55c-4372-aade-4c21ba8bf6e3 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:17.040930 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 491/986] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:17.047713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3102466-f09e-4dd6-bf3b-c38d98283741 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:17.163101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3102466-f09e-4dd6-bf3b-c38d98283741 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:17.194672 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:17.204158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db459e-3786-4b59-ae98-762661b058e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:17.205503 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db459e-3786-4b59-ae98-762661b058e6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:17.205992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db459e-3786-4b59-ae98-762661b058e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:17.206498 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db459e-3786-4b59-ae98-762661b058e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:17.214153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db459e-3786-4b59-ae98-762661b058e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:17.215884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db459e-3786-4b59-ae98-762661b058e6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:17.230835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-07db459e-3786-4b59-ae98-762661b058e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:17.230835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:17.230835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:17.230835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:17.230835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:17.230835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:17.230835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:17.230835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:17.230835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:17.230835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:17.230835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:17.230835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:17.230835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:57:17.230835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:57:17.230835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:17.231745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:17.232666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:57:17.232666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:57:17.232666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:57:17.232666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:57:17.232666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:57:17.232666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:57:17.232666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:17.232666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:17.238011 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 496/987] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:57:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:17.239572 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878972b5-c55c-4372-aade-4c21ba8bf6e3 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:17.349938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878972b5-c55c-4372-aade-4c21ba8bf6e3 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:17.388146 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880702b-fe09-44c2-899e-6da31cc6d3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:17.389216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880702b-fe09-44c2-899e-6da31cc6d3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:17.389701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880702b-fe09-44c2-899e-6da31cc6d3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:17.390184 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880702b-fe09-44c2-899e-6da31cc6d3f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:17.391729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880702b-fe09-44c2-899e-6da31cc6d3f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:17.392263 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880702b-fe09-44c2-899e-6da31cc6d3f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:17.394052 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880702b-fe09-44c2-899e-6da31cc6d3f1 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:17.406995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 492/988] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:57:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:17.413179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db459e-3786-4b59-ae98-762661b058e6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:17.519714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db459e-3786-4b59-ae98-762661b058e6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:17.555014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd3c6e-59e4-4764-89e0-e0d828e5fda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:17.555670 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd3c6e-59e4-4764-89e0-e0d828e5fda1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:17.555976 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd3c6e-59e4-4764-89e0-e0d828e5fda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:17.556257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd3c6e-59e4-4764-89e0-e0d828e5fda1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:17.557089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd3c6e-59e4-4764-89e0-e0d828e5fda1 None admin] RBAC: Adding query filter params (role.id=1d3b684bd8714b5792ebd7695c02a9ab, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=257450db238d4efda783e30cbcd23d4f) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:17.557398 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd3c6e-59e4-4764-89e0-e0d828e5fda1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:17.558836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd3c6e-59e4-4764-89e0-e0d828e5fda1 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:17.565038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 497/989] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:57:17 2022] GET /identity/v3/role_assignments?user.id=257450db238d4efda783e30cbcd23d4f&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=1d3b684bd8714b5792ebd7695c02a9ab => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:18.955320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9997c-6c2a-448c-8c24-9af333880bf6 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:18.955870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9997c-6c2a-448c-8c24-9af333880bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:18.955870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9997c-6c2a-448c-8c24-9af333880bf6 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:18.957301 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 493/990] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:18.962450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6eda01-ae8f-4264-bfe6-1f4711846b77 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:18.962839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6eda01-ae8f-4264-bfe6-1f4711846b77 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:18.962999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6eda01-ae8f-4264-bfe6-1f4711846b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:18.980079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ed6eda01-ae8f-4264-bfe6-1f4711846b77 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:18.981188 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ed6eda01-ae8f-4264-bfe6-1f4711846b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:19.030279 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:19.052557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 498/991] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:19.074876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e9997c-6c2a-448c-8c24-9af333880bf6 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:19.077801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:19.143961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda883fb-1565-4d4d-bb66-c3eb323221ce None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:19.144224 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda883fb-1565-4d4d-bb66-c3eb323221ce None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:19.144666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda883fb-1565-4d4d-bb66-c3eb323221ce None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:19.146210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 499/992] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:57:19.151381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c2009-76f3-4893-8b4f-ebd6f8f8b387 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:19.151682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c2009-76f3-4893-8b4f-ebd6f8f8b387 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:19.151996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c2009-76f3-4893-8b4f-ebd6f8f8b387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:19.171105 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d38c2009-76f3-4893-8b4f-ebd6f8f8b387 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:19.172663 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d38c2009-76f3-4893-8b4f-ebd6f8f8b387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:19.200114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e9997c-6c2a-448c-8c24-9af333880bf6 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:19.242884 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056556d-7d0d-434b-86d0-0641c8033820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:19.243600 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056556d-7d0d-434b-86d0-0641c8033820 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:19.243887 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056556d-7d0d-434b-86d0-0641c8033820 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:19.244194 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056556d-7d0d-434b-86d0-0641c8033820 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:19.244858 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056556d-7d0d-434b-86d0-0641c8033820 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:19.246247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056556d-7d0d-434b-86d0-0641c8033820 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:19.250839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 500/993] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-5056556d-7d0d-434b-86d0-0641c8033820 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:19.258714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:19.259699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:19.260876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:19.260876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:19.260876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:19.260876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:19.260876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:19.260876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:19.260876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:19.260876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:19.260876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:19.260876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:19.260876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:19.260876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:19.260876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:57:19.260876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:19.260876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 494/994] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:57:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:19.266875 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c2009-76f3-4893-8b4f-ebd6f8f8b387 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:19.277307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056556d-7d0d-434b-86d0-0641c8033820 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:19.357614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:19.386857 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c2009-76f3-4893-8b4f-ebd6f8f8b387 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:19.401024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056556d-7d0d-434b-86d0-0641c8033820 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:19.404199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:19.429394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d34bb-2548-488f-baf2-1a2eeb8bc057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:19.430066 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d34bb-2548-488f-baf2-1a2eeb8bc057 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:19.430350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d34bb-2548-488f-baf2-1a2eeb8bc057 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:19.430625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d34bb-2548-488f-baf2-1a2eeb8bc057 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:19.431334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d34bb-2548-488f-baf2-1a2eeb8bc057 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:19.431639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d34bb-2548-488f-baf2-1a2eeb8bc057 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:19.433055 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d34bb-2548-488f-baf2-1a2eeb8bc057 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:19.439172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 501/995] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jul 21 08:57:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:19.442061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee479a8-e713-4cee-be09-2eab6187bf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:19.442878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee479a8-e713-4cee-be09-2eab6187bf8c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:19.443172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee479a8-e713-4cee-be09-2eab6187bf8c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:19.443447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee479a8-e713-4cee-be09-2eab6187bf8c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:19.444238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee479a8-e713-4cee-be09-2eab6187bf8c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:19.445565 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d34bb-2548-488f-baf2-1a2eeb8bc057 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:19.445750 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee479a8-e713-4cee-be09-2eab6187bf8c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-3ee479a8-e713-4cee-be09-2eab6187bf8c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:19.460716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:19.461773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:19.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:19.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:19.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:19.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:19.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:19.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:19.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:19.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:19.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:19.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:19.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:19.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:19.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:57:19.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:19.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 495/996] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jul 21 08:57:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:19.467812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee479a8-e713-4cee-be09-2eab6187bf8c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:19.560701 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d34bb-2548-488f-baf2-1a2eeb8bc057 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:19.590523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee479a8-e713-4cee-be09-2eab6187bf8c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:19.597722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ab4b2-67d1-431f-9b0d-0d1ce517fdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:19.598381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ab4b2-67d1-431f-9b0d-0d1ce517fdf2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:19.598666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ab4b2-67d1-431f-9b0d-0d1ce517fdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:19.598936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ab4b2-67d1-431f-9b0d-0d1ce517fdf2 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:19.599634 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ab4b2-67d1-431f-9b0d-0d1ce517fdf2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:19.599950 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ab4b2-67d1-431f-9b0d-0d1ce517fdf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:19.601321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ab4b2-67d1-431f-9b0d-0d1ce517fdf2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:19.606189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 502/997] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jul 21 08:57:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:19.612520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ab4b2-67d1-431f-9b0d-0d1ce517fdf2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:19.634312 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:19.638218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff8f61-8a11-40f4-b430-817b58c3fe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:19.638907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff8f61-8a11-40f4-b430-817b58c3fe64 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:19.639222 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff8f61-8a11-40f4-b430-817b58c3fe64 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:19.639494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff8f61-8a11-40f4-b430-817b58c3fe64 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:19.640344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff8f61-8a11-40f4-b430-817b58c3fe64 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:19.642350 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff8f61-8a11-40f4-b430-817b58c3fe64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:19.644681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff8f61-8a11-40f4-b430-817b58c3fe64 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:19.650052 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 496/998] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jul 21 08:57:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:19.655474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff8f61-8a11-40f4-b430-817b58c3fe64 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:19.664833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:19.732643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ab4b2-67d1-431f-9b0d-0d1ce517fdf2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:19.761151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff8f61-8a11-40f4-b430-817b58c3fe64 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:19.767400 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eb5d9-2dbc-42d5-bea3-233a32e92a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:19.768092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eb5d9-2dbc-42d5-bea3-233a32e92a64 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:19.768377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eb5d9-2dbc-42d5-bea3-233a32e92a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:19.768647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eb5d9-2dbc-42d5-bea3-233a32e92a64 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:19.769434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eb5d9-2dbc-42d5-bea3-233a32e92a64 None admin] RBAC: Adding query filter params (interface=public, service_id=95e5a35ec2314b4cbe3dcd8abbc6b723, region_id=RegionOne) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:19.769740 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eb5d9-2dbc-42d5-bea3-233a32e92a64 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:19.771103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eb5d9-2dbc-42d5-bea3-233a32e92a64 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:19.776278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 503/999] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jul 21 08:57:19 2022] GET /identity/v3/endpoints?service_id=95e5a35ec2314b4cbe3dcd8abbc6b723&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:19.777505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fcd47-6e28-4bd1-9253-2cc421536d2e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:19.777794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fcd47-6e28-4bd1-9253-2cc421536d2e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:19.778052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fcd47-6e28-4bd1-9253-2cc421536d2e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:19.779430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 504/1000] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:19.786695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fcd47-6e28-4bd1-9253-2cc421536d2e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:19.803022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bcb41-6d72-4be9-871b-16776c675957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:19.803678 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bcb41-6d72-4be9-871b-16776c675957 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:19.803990 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bcb41-6d72-4be9-871b-16776c675957 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:19.804265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bcb41-6d72-4be9-871b-16776c675957 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:19.804983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bcb41-6d72-4be9-871b-16776c675957 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:19.805292 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bcb41-6d72-4be9-871b-16776c675957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:19.806683 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bcb41-6d72-4be9-871b-16776c675957 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:19.811908 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 497/1001] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jul 21 08:57:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:19.813228 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2440fb-0a29-4897-986f-594a7d156194 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:19.813520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2440fb-0a29-4897-986f-594a7d156194 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:19.813794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2440fb-0a29-4897-986f-594a7d156194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:19.832674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4f2440fb-0a29-4897-986f-594a7d156194 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:19.833885 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4f2440fb-0a29-4897-986f-594a7d156194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:19.893089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fcd47-6e28-4bd1-9253-2cc421536d2e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:19.895612 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:19.900822 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:19.916737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 498/1002] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:19.917363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2440fb-0a29-4897-986f-594a7d156194 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:19.944089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f84c7b-bc7f-4173-887d-bda89ecb765c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:19.944770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f84c7b-bc7f-4173-887d-bda89ecb765c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:19.945058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f84c7b-bc7f-4173-887d-bda89ecb765c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:19.945341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f84c7b-bc7f-4173-887d-bda89ecb765c None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:19.946038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f84c7b-bc7f-4173-887d-bda89ecb765c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:19.947426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f84c7b-bc7f-4173-887d-bda89ecb765c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:19.954376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 505/1003] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jul 21 08:57:19 2022] GET /identity/v3/services => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:57:19.955203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f84c7b-bc7f-4173-887d-bda89ecb765c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:20.034480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2440fb-0a29-4897-986f-594a7d156194 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:20.081282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3ed06-d991-4168-a24d-9290402d7b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:20.081883 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3ed06-d991-4168-a24d-9290402d7b75 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:20.082172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3ed06-d991-4168-a24d-9290402d7b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:20.082450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3ed06-d991-4168-a24d-9290402d7b75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:20.083216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3ed06-d991-4168-a24d-9290402d7b75 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:20.084821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3ed06-d991-4168-a24d-9290402d7b75 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:20.094925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f84c7b-bc7f-4173-887d-bda89ecb765c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:20.103508 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 499/1004] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jul 21 08:57:19 2022] POST /identity/v3/endpoints => generated 325 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:57:20.147503 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ccc87-5d8b-4d75-86b5-abbd97496abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:20.148352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ccc87-5d8b-4d75-86b5-abbd97496abe None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:20.148638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ccc87-5d8b-4d75-86b5-abbd97496abe None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:20.148912 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ccc87-5d8b-4d75-86b5-abbd97496abe None admin] PATH_INFO: `/v3/services/image` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:20.149571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ccc87-5d8b-4d75-86b5-abbd97496abe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:20.150989 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ccc87-5d8b-4d75-86b5-abbd97496abe None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-b19ccc87-5d8b-4d75-86b5-abbd97496abe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:20.163993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:20.165332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:20.166402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:20.166402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:20.166402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:20.166402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:20.166402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:20.166402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:20.166402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:20.166402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:20.166402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:20.166402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:20.166402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:20.166402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:20.166402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:57:20.166402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:20.166402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 506/1005] 10.222.0.13 () {56 vars in 1105 bytes} [Thu Jul 21 08:57:19 2022] GET /identity/v3/services/image => generated 86 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:20.171403 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3ed06-d991-4168-a24d-9290402d7b75 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:20.228825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:20.294344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3ed06-d991-4168-a24d-9290402d7b75 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:20.334029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3199b-2be2-4683-9fda-ec0f94297457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:20.334231 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3199b-2be2-4683-9fda-ec0f94297457 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:20.334459 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3199b-2be2-4683-9fda-ec0f94297457 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:20.335151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3199b-2be2-4683-9fda-ec0f94297457 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:20.335746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3199b-2be2-4683-9fda-ec0f94297457 None admin] RBAC: Adding query filter params (name=image) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:20.335746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3199b-2be2-4683-9fda-ec0f94297457 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:20.337134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3199b-2be2-4683-9fda-ec0f94297457 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:20.342355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 500/1006] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:20.348638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ccc87-5d8b-4d75-86b5-abbd97496abe None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:20.379881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:20.466318 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ccc87-5d8b-4d75-86b5-abbd97496abe None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:20.502653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b9438-03a9-47d9-b7c8-587e724344d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:20.503369 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b9438-03a9-47d9-b7c8-587e724344d6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:20.503655 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b9438-03a9-47d9-b7c8-587e724344d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:20.503937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b9438-03a9-47d9-b7c8-587e724344d6 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:20.504655 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b9438-03a9-47d9-b7c8-587e724344d6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:20.504955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b9438-03a9-47d9-b7c8-587e724344d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:20.506322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b9438-03a9-47d9-b7c8-587e724344d6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:20.519570 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 507/1007] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:20 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:22.052830 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0f5a6-5579-4ba0-92bd-0a9429052301 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:22.053343 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0f5a6-5579-4ba0-92bd-0a9429052301 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:22.053343 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0f5a6-5579-4ba0-92bd-0a9429052301 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:22.054705 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 501/1008] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:22.059890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b943d-7787-45bd-b34f-ad947a345f1d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:22.060325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b943d-7787-45bd-b34f-ad947a345f1d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:22.060512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b943d-7787-45bd-b34f-ad947a345f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:22.077578 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-476b943d-7787-45bd-b34f-ad947a345f1d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:22.079310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-476b943d-7787-45bd-b34f-ad947a345f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:22.150773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 508/1009] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:22.174026 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0f5a6-5579-4ba0-92bd-0a9429052301 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:22.246870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bae29-b48b-4af1-bd3a-67463de39bf8 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:22.247205 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bae29-b48b-4af1-bd3a-67463de39bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:22.247421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5bae29-b48b-4af1-bd3a-67463de39bf8 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:22.248966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 509/1010] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:57:22.254671 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba28d9-3ea9-4f67-8e45-a7f86bcd190b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:22.254961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba28d9-3ea9-4f67-8e45-a7f86bcd190b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:22.255235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba28d9-3ea9-4f67-8e45-a7f86bcd190b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:22.272238 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-daba28d9-3ea9-4f67-8e45-a7f86bcd190b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:22.273440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-daba28d9-3ea9-4f67-8e45-a7f86bcd190b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:22.281873 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0f5a6-5579-4ba0-92bd-0a9429052301 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:22.290984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:22.296184 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:22.323734 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409bcb1-a5ba-43b9-aa15-4314261b4629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:22.324498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409bcb1-a5ba-43b9-aa15-4314261b4629 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:22.324776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409bcb1-a5ba-43b9-aa15-4314261b4629 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:22.325061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409bcb1-a5ba-43b9-aa15-4314261b4629 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:22.325744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409bcb1-a5ba-43b9-aa15-4314261b4629 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:22.327201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409bcb1-a5ba-43b9-aa15-4314261b4629 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-f409bcb1-a5ba-43b9-aa15-4314261b4629 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:22.340195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:22.341272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:22.342363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:22.342363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:22.342363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:22.342363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:22.342363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:22.342363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:22.342363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:22.342363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:22.342363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:22.342363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:22.342363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:22.342363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:22.342363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:57:22.342363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:22.342363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 502/1011] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:57:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:22.347880 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409bcb1-a5ba-43b9-aa15-4314261b4629 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:22.356430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 510/1012] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:22.380808 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba28d9-3ea9-4f67-8e45-a7f86bcd190b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:22.465510 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409bcb1-a5ba-43b9-aa15-4314261b4629 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:22.504915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ae9c5-f94f-4dbd-bd10-ceaefd1651da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:22.505605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ae9c5-f94f-4dbd-bd10-ceaefd1651da None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:22.505892 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ae9c5-f94f-4dbd-bd10-ceaefd1651da None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:22.506257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ae9c5-f94f-4dbd-bd10-ceaefd1651da None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:22.506434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba28d9-3ea9-4f67-8e45-a7f86bcd190b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:22.507001 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ae9c5-f94f-4dbd-bd10-ceaefd1651da None admin] RBAC: Adding query filter params (name=compute) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:22.507316 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ae9c5-f94f-4dbd-bd10-ceaefd1651da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:22.508778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ae9c5-f94f-4dbd-bd10-ceaefd1651da None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:22.514446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 503/1013] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jul 21 08:57:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:22.520562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ae9c5-f94f-4dbd-bd10-ceaefd1651da None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:22.547824 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74281e71-9538-496c-a499-2c925f764e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:22.548660 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74281e71-9538-496c-a499-2c925f764e39 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:22.548946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74281e71-9538-496c-a499-2c925f764e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:22.549220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74281e71-9538-496c-a499-2c925f764e39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:22.550546 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74281e71-9538-496c-a499-2c925f764e39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:22.551948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74281e71-9538-496c-a499-2c925f764e39 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:22.553816 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 511/1014] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:22.559668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74281e71-9538-496c-a499-2c925f764e39 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:22.568322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:22.587477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:22.637524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ae9c5-f94f-4dbd-bd10-ceaefd1651da None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:22.674376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74281e71-9538-496c-a499-2c925f764e39 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:22.674621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619f284-537a-4021-bfbb-400372edb338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:22.675081 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619f284-537a-4021-bfbb-400372edb338 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:22.675385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619f284-537a-4021-bfbb-400372edb338 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:22.675660 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619f284-537a-4021-bfbb-400372edb338 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:22.676406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619f284-537a-4021-bfbb-400372edb338 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:22.676718 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619f284-537a-4021-bfbb-400372edb338 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:22.678344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619f284-537a-4021-bfbb-400372edb338 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:22.686619 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 504/1015] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jul 21 08:57:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:22.688266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd974a4-7134-4243-b288-57d73abc9b56 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:22.688586 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd974a4-7134-4243-b288-57d73abc9b56 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:22.688865 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd974a4-7134-4243-b288-57d73abc9b56 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:22.690127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 505/1016] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:22.692885 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd974a4-7134-4243-b288-57d73abc9b56 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:22.717358 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7548af-d1d9-410a-b7ff-e56d9a10178c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:22.718028 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7548af-d1d9-410a-b7ff-e56d9a10178c None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:22.718301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7548af-d1d9-410a-b7ff-e56d9a10178c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:22.718583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7548af-d1d9-410a-b7ff-e56d9a10178c None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:22.720560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7548af-d1d9-410a-b7ff-e56d9a10178c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:22.722190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7548af-d1d9-410a-b7ff-e56d9a10178c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:22.759296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 512/1017] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jul 21 08:57:22 2022] POST /identity/v3/projects => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:57:22.759296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6912e-bf82-49d8-81c9-0f5812e333a3 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:22.759296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6912e-bf82-49d8-81c9-0f5812e333a3 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:22.759296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6912e-bf82-49d8-81c9-0f5812e333a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:22.777288 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-dcd6912e-bf82-49d8-81c9-0f5812e333a3 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:22.778551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-dcd6912e-bf82-49d8-81c9-0f5812e333a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:22.811083 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd974a4-7134-4243-b288-57d73abc9b56 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:22.814773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:22.852689 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5e4c9-3ff3-4227-8dd4-7453eeacf446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:22.853336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5e4c9-3ff3-4227-8dd4-7453eeacf446 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:22.853625 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5e4c9-3ff3-4227-8dd4-7453eeacf446 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:22.853930 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5e4c9-3ff3-4227-8dd4-7453eeacf446 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:22.854723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5e4c9-3ff3-4227-8dd4-7453eeacf446 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:22.856302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5e4c9-3ff3-4227-8dd4-7453eeacf446 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:22.864004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 513/1018] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:22 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:22.882098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6912e-bf82-49d8-81c9-0f5812e333a3 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:22.884418 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:22.886220 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 506/1019] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jul 21 08:57:22 2022] POST /identity/v3/endpoints => generated 334 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:57:23.012815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6912e-bf82-49d8-81c9-0f5812e333a3 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:23.061725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796aa9c2-4663-4046-8e98-e08d25964102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:23.062472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796aa9c2-4663-4046-8e98-e08d25964102 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:23.063417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796aa9c2-4663-4046-8e98-e08d25964102 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:23.064466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796aa9c2-4663-4046-8e98-e08d25964102 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:23.064466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796aa9c2-4663-4046-8e98-e08d25964102 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:23.065859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796aa9c2-4663-4046-8e98-e08d25964102 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:23.082284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 514/1020] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jul 21 08:57:22 2022] POST /identity/v3/services => generated 242 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:57:24.894849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e1e5a-385f-48db-9454-6292903abc01 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:24.895409 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e1e5a-385f-48db-9454-6292903abc01 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:24.895409 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29e1e5a-385f-48db-9454-6292903abc01 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:24.896835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 507/1021] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:24.902328 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4a973-6c72-491a-a598-fc257a9919ec None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:24.902765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4a973-6c72-491a-a598-fc257a9919ec None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:24.902956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4a973-6c72-491a-a598-fc257a9919ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:24.922321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-63d4a973-6c72-491a-a598-fc257a9919ec None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:24.923788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-63d4a973-6c72-491a-a598-fc257a9919ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:24.980988 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:25.008090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 515/1022] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:25.029684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e1e5a-385f-48db-9454-6292903abc01 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:25.096056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7918189-1d54-441b-b378-ae067665db45 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:25.096349 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7918189-1d54-441b-b378-ae067665db45 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:25.096627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7918189-1d54-441b-b378-ae067665db45 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:25.098013 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 516/1023] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:57:25.103394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a654f-fbc0-4102-9749-a0d0bfa4a4ed None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:25.103685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a654f-fbc0-4102-9749-a0d0bfa4a4ed None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:25.103998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a654f-fbc0-4102-9749-a0d0bfa4a4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:25.121757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-fc2a654f-fbc0-4102-9749-a0d0bfa4a4ed None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:25.123361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fc2a654f-fbc0-4102-9749-a0d0bfa4a4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:25.149736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29e1e5a-385f-48db-9454-6292903abc01 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:25.193045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8754b7-9552-4991-8fd2-533e836742a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:25.193834 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8754b7-9552-4991-8fd2-533e836742a0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:25.194114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8754b7-9552-4991-8fd2-533e836742a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:25.194391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8754b7-9552-4991-8fd2-533e836742a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:25.195599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8754b7-9552-4991-8fd2-533e836742a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:25.197093 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8754b7-9552-4991-8fd2-533e836742a0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:25.198094 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 517/1024] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:25.199411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e134e-95fd-410b-9dac-d947d9c491c8 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:25.199701 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e134e-95fd-410b-9dac-d947d9c491c8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:25.200007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e134e-95fd-410b-9dac-d947d9c491c8 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:25.205565 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 508/1025] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:25.205718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 518/1026] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:25.206739 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8754b7-9552-4991-8fd2-533e836742a0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:25.209407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73568365-5ecc-450c-9957-901d61070900 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:25.209697 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73568365-5ecc-450c-9957-901d61070900 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:25.209976 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73568365-5ecc-450c-9957-901d61070900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:25.227330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-73568365-5ecc-450c-9957-901d61070900 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:25.228613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-73568365-5ecc-450c-9957-901d61070900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:25.236129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:25.298369 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 519/1027] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:25.299539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73568365-5ecc-450c-9957-901d61070900 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:25.314440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8754b7-9552-4991-8fd2-533e836742a0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:25.347132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:25.349441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312d47f-601f-4867-bf63-695dd1140080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:25.350260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312d47f-601f-4867-bf63-695dd1140080 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:25.350551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312d47f-601f-4867-bf63-695dd1140080 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:25.350825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312d47f-601f-4867-bf63-695dd1140080 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:25.351833 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312d47f-601f-4867-bf63-695dd1140080 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:25.353311 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312d47f-601f-4867-bf63-695dd1140080 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:25.374214 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 509/1028] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:57:25 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:57:25.375623 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312d47f-601f-4867-bf63-695dd1140080 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:25.411890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73568365-5ecc-450c-9957-901d61070900 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:25.454492 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da25c87-bd8e-4ecf-8309-91a7bb998bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:25.455336 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da25c87-bd8e-4ecf-8309-91a7bb998bfe None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:25.455619 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da25c87-bd8e-4ecf-8309-91a7bb998bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:25.455897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da25c87-bd8e-4ecf-8309-91a7bb998bfe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:25.468496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da25c87-bd8e-4ecf-8309-91a7bb998bfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-5da25c87-bd8e-4ecf-8309-91a7bb998bfe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:57:25.478528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:25.479616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:25.480960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:25.480960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:25.480960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:25.480960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:25.480960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:57:25.480960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:57:25.480960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:57:25.480960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:57:25.480960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:57:25.480960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:57:25.480960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:57:25.480960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:25.480960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 520/1029] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jul 21 08:57:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:25.487257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da25c87-bd8e-4ecf-8309-91a7bb998bfe None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:25.490534 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:25.512633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312d47f-601f-4867-bf63-695dd1140080 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:25.556495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3c9cd-592f-4db7-a310-f353a35a31c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:25.557308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3c9cd-592f-4db7-a310-f353a35a31c7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:25.557587 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3c9cd-592f-4db7-a310-f353a35a31c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:25.557860 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3c9cd-592f-4db7-a310-f353a35a31c7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:25.558521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3c9cd-592f-4db7-a310-f353a35a31c7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:25.559930 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3c9cd-592f-4db7-a310-f353a35a31c7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-16f3c9cd-592f-4db7-a310-f353a35a31c7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:25.572481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:25.573492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:25.574359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:25.574359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:25.574359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:25.574359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:25.574359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:25.574359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:25.574359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:25.574359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:25.574359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:25.574359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:25.574359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:25.574359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:25.574359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:57:25.574359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:25.574359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 510/1030] 10.222.0.13 () {56 vars in 1105 bytes} [Thu Jul 21 08:57:25 2022] GET /identity/v3/services/image => generated 86 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:25.580411 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3c9cd-592f-4db7-a310-f353a35a31c7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:25.615531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da25c87-bd8e-4ecf-8309-91a7bb998bfe None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:25.617968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:25.654629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2938c7-3f91-4e76-9660-33ce15f3cb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:25.655588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2938c7-3f91-4e76-9660-33ce15f3cb9b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:25.656098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2938c7-3f91-4e76-9660-33ce15f3cb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:25.656552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2938c7-3f91-4e76-9660-33ce15f3cb9b None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:25.657414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2938c7-3f91-4e76-9660-33ce15f3cb9b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:25.657726 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2938c7-3f91-4e76-9660-33ce15f3cb9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:25.659120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2938c7-3f91-4e76-9660-33ce15f3cb9b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:25.667948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 521/1031] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jul 21 08:57:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:25.674403 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2938c7-3f91-4e76-9660-33ce15f3cb9b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:25.700375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3c9cd-592f-4db7-a310-f353a35a31c7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:25.742898 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25daa62-8432-4d33-98cc-2b9c2fcf3db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:25.743488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25daa62-8432-4d33-98cc-2b9c2fcf3db9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:25.743768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25daa62-8432-4d33-98cc-2b9c2fcf3db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:25.744073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25daa62-8432-4d33-98cc-2b9c2fcf3db9 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:25.744781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25daa62-8432-4d33-98cc-2b9c2fcf3db9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:25.745084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25daa62-8432-4d33-98cc-2b9c2fcf3db9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:25.746474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25daa62-8432-4d33-98cc-2b9c2fcf3db9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:25.752580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 511/1032] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:25.758129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25daa62-8432-4d33-98cc-2b9c2fcf3db9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:25.774049 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:25.788377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2938c7-3f91-4e76-9660-33ce15f3cb9b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:25.827874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f1f50-cf8c-41fd-aa4e-bd961028d8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:25.829288 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f1f50-cf8c-41fd-aa4e-bd961028d8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:25.829713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f1f50-cf8c-41fd-aa4e-bd961028d8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:25.829992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f1f50-cf8c-41fd-aa4e-bd961028d8a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:25.845524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f1f50-cf8c-41fd-aa4e-bd961028d8a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:25.847538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f1f50-cf8c-41fd-aa4e-bd961028d8a7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:25.868165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-b84f1f50-cf8c-41fd-aa4e-bd961028d8a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:25.868165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:25.868165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:25.868165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:25.868165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:25.868165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:25.868165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:25.868165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:25.868165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:25.868165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:25.868165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:25.868165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:25.868165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:25.868165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:25.868165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:25.869343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:25.870275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:25.870275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:25.870275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:25.870275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:25.870275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 522/1033] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:25.876158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f1f50-cf8c-41fd-aa4e-bd961028d8a7 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:25.881288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25daa62-8432-4d33-98cc-2b9c2fcf3db9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:25.897629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:25.918539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0790fcf-5d1d-4507-8447-f8ac9b48829a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:25.919211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0790fcf-5d1d-4507-8447-f8ac9b48829a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:25.919496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0790fcf-5d1d-4507-8447-f8ac9b48829a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:25.919774 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0790fcf-5d1d-4507-8447-f8ac9b48829a None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:25.920524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0790fcf-5d1d-4507-8447-f8ac9b48829a None admin] RBAC: Adding query filter params (type=image) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:25.920832 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0790fcf-5d1d-4507-8447-f8ac9b48829a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:25.923166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0790fcf-5d1d-4507-8447-f8ac9b48829a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:25.928853 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 512/1034] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:25.935110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0790fcf-5d1d-4507-8447-f8ac9b48829a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:25.996548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f1f50-cf8c-41fd-aa4e-bd961028d8a7 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:26.033457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:26.033707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd533bd-7bf4-4f95-a20c-1c5a7cdbcb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:26.034377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd533bd-7bf4-4f95-a20c-1c5a7cdbcb02 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:26.034663 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd533bd-7bf4-4f95-a20c-1c5a7cdbcb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:26.034949 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd533bd-7bf4-4f95-a20c-1c5a7cdbcb02 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:26.035674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd533bd-7bf4-4f95-a20c-1c5a7cdbcb02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:26.036039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd533bd-7bf4-4f95-a20c-1c5a7cdbcb02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:26.037419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd533bd-7bf4-4f95-a20c-1c5a7cdbcb02 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:26.051249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 523/1035] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:26.056725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd533bd-7bf4-4f95-a20c-1c5a7cdbcb02 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:26.060124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0790fcf-5d1d-4507-8447-f8ac9b48829a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:26.097088 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428709c7-6d46-45e6-a6aa-b52649282617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:26.097760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428709c7-6d46-45e6-a6aa-b52649282617 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:26.098050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428709c7-6d46-45e6-a6aa-b52649282617 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:26.098330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428709c7-6d46-45e6-a6aa-b52649282617 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:26.099130 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428709c7-6d46-45e6-a6aa-b52649282617 None admin] RBAC: Adding query filter params (interface=public, service_id=2b4241aaa145473481e6848b62b3b733, region_id=RegionOne) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:26.099446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428709c7-6d46-45e6-a6aa-b52649282617 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:26.100888 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428709c7-6d46-45e6-a6aa-b52649282617 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:26.107244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 513/1036] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jul 21 08:57:25 2022] GET /identity/v3/endpoints?service_id=2b4241aaa145473481e6848b62b3b733&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:26.113772 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428709c7-6d46-45e6-a6aa-b52649282617 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:26.146383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:26.171171 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd533bd-7bf4-4f95-a20c-1c5a7cdbcb02 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:26.208849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67110c66-184c-4d85-833e-1f3c047311a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:26.209644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67110c66-184c-4d85-833e-1f3c047311a7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:26.209928 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67110c66-184c-4d85-833e-1f3c047311a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:26.210203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67110c66-184c-4d85-833e-1f3c047311a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:26.217977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67110c66-184c-4d85-833e-1f3c047311a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:26.220094 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67110c66-184c-4d85-833e-1f3c047311a7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:26.228681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428709c7-6d46-45e6-a6aa-b52649282617 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-67110c66-184c-4d85-833e-1f3c047311a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:57:26.235765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:26.236786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:26.237743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:26.237743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:26.237743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:57:26.237743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:57:26.237743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:57:26.237743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:57:26.237743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:57:26.237743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:57:26.237743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:57:26.237743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:26.237743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 524/1037] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jul 21 08:57:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:26.242685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67110c66-184c-4d85-833e-1f3c047311a7 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:26.269476 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2610ad-fe1a-4b84-9705-9a3ad98eaf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:26.270047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2610ad-fe1a-4b84-9705-9a3ad98eaf9f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:26.270329 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2610ad-fe1a-4b84-9705-9a3ad98eaf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:26.270607 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2610ad-fe1a-4b84-9705-9a3ad98eaf9f None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:26.271283 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2610ad-fe1a-4b84-9705-9a3ad98eaf9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:26.272695 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2610ad-fe1a-4b84-9705-9a3ad98eaf9f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:26.279308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 514/1038] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jul 21 08:57:26 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:57:26.353514 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67110c66-184c-4d85-833e-1f3c047311a7 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:26.392114 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c61d95-e828-4a83-9da3-ed78df66cdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:26.392812 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c61d95-e828-4a83-9da3-ed78df66cdd6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:26.393102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c61d95-e828-4a83-9da3-ed78df66cdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:26.393382 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c61d95-e828-4a83-9da3-ed78df66cdd6 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:26.394248 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c61d95-e828-4a83-9da3-ed78df66cdd6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:26.394561 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c61d95-e828-4a83-9da3-ed78df66cdd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:26.396080 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c61d95-e828-4a83-9da3-ed78df66cdd6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:26.405218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:26.418042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 525/1039] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jul 21 08:57:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:26.424676 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2610ad-fe1a-4b84-9705-9a3ad98eaf9f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:26.428864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:26.536033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2610ad-fe1a-4b84-9705-9a3ad98eaf9f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:26.570844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd1d0c-3ea1-4836-809d-0b5f87a94f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:26.571648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd1d0c-3ea1-4836-809d-0b5f87a94f08 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:26.572070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd1d0c-3ea1-4836-809d-0b5f87a94f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:26.572454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd1d0c-3ea1-4836-809d-0b5f87a94f08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:26.649420 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd1d0c-3ea1-4836-809d-0b5f87a94f08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:26.651082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd1d0c-3ea1-4836-809d-0b5f87a94f08 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:26.654040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:26.768082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 515/1040] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:57:26 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:57:26.770637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c61d95-e828-4a83-9da3-ed78df66cdd6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:26.881554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c61d95-e828-4a83-9da3-ed78df66cdd6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:26.916826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b812fa7-54e9-4060-81f8-09b6ec79a2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:26.917896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b812fa7-54e9-4060-81f8-09b6ec79a2ab None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:26.918331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b812fa7-54e9-4060-81f8-09b6ec79a2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:26.918775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b812fa7-54e9-4060-81f8-09b6ec79a2ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:26.930247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b812fa7-54e9-4060-81f8-09b6ec79a2ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:26.931954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b812fa7-54e9-4060-81f8-09b6ec79a2ab None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:26.950763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-3b812fa7-54e9-4060-81f8-09b6ec79a2ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:26.950763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:26.950763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:26.950763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:26.950763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:26.950763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:26.950763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:26.950763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:26.950763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:26.950763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:26.950763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:26.950763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:26.950763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:26.950763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:26.950763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:26.951785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:26.956249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:26.956249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:26.956249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:26.956249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:26.956249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 526/1041] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:26.959793 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd1d0c-3ea1-4836-809d-0b5f87a94f08 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:27.066569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd1d0c-3ea1-4836-809d-0b5f87a94f08 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:27.069377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:27.105140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a364f-028a-4e19-b026-506b309de8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:27.106113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a364f-028a-4e19-b026-506b309de8ef None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:27.106579 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a364f-028a-4e19-b026-506b309de8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:27.107065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a364f-028a-4e19-b026-506b309de8ef None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:27.108057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a364f-028a-4e19-b026-506b309de8ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:27.108574 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a364f-028a-4e19-b026-506b309de8ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:27.110293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a364f-028a-4e19-b026-506b309de8ef None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:27.126853 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 516/1042] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:27.129744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b812fa7-54e9-4060-81f8-09b6ec79a2ab None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:27.183524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:27.238928 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b812fa7-54e9-4060-81f8-09b6ec79a2ab None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:27.273909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccc972-c121-4a86-a73b-86f026646386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:27.274706 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccc972-c121-4a86-a73b-86f026646386 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:27.274990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccc972-c121-4a86-a73b-86f026646386 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:27.275266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccc972-c121-4a86-a73b-86f026646386 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:27.282837 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccc972-c121-4a86-a73b-86f026646386 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:27.284335 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccc972-c121-4a86-a73b-86f026646386 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:27.299508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-30ccc972-c121-4a86-a73b-86f026646386 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:27.299508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:27.299508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:27.299508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:27.299508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:27.299508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:27.299508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:27.299508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:27.299508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:27.299508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:27.299508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:27.299508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:27.299508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:57:27.299508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:57:27.299508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:27.300630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:27.301522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:57:27.301522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:57:27.301522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:57:27.301522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:57:27.301522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:57:27.301522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:57:27.301522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:27.301522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:27.304274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 527/1043] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:57:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:27.306558 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a364f-028a-4e19-b026-506b309de8ef None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:27.413488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a364f-028a-4e19-b026-506b309de8ef None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:27.455159 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40a330-494f-405a-bec4-ae1f5b80f616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:27.455760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40a330-494f-405a-bec4-ae1f5b80f616 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:27.456084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40a330-494f-405a-bec4-ae1f5b80f616 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:27.456363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40a330-494f-405a-bec4-ae1f5b80f616 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:27.457170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40a330-494f-405a-bec4-ae1f5b80f616 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:27.457478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40a330-494f-405a-bec4-ae1f5b80f616 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:27.459052 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40a330-494f-405a-bec4-ae1f5b80f616 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:27.479000 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 517/1044] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:57:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:27.480451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccc972-c121-4a86-a73b-86f026646386 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:27.527048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192cfc4-d576-4bcb-a886-06b3e8232475 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:27.527595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192cfc4-d576-4bcb-a886-06b3e8232475 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:27.527874 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192cfc4-d576-4bcb-a886-06b3e8232475 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:27.529352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 518/1045] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:57:27.535396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5afaae-77fe-4bd4-986c-523790988ab3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:27.535685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5afaae-77fe-4bd4-986c-523790988ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:27.536017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5afaae-77fe-4bd4-986c-523790988ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:27.554124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ac5afaae-77fe-4bd4-986c-523790988ab3 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:27.555335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ac5afaae-77fe-4bd4-986c-523790988ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:27.576852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:27.606174 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccc972-c121-4a86-a73b-86f026646386 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:27.617710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:27.642966 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 519/1046] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:27.654162 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916431b-ff9a-4aaf-8ee2-dfb657fe1690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:27.654839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916431b-ff9a-4aaf-8ee2-dfb657fe1690 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:27.655120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916431b-ff9a-4aaf-8ee2-dfb657fe1690 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:27.655394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916431b-ff9a-4aaf-8ee2-dfb657fe1690 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:27.656326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916431b-ff9a-4aaf-8ee2-dfb657fe1690 None admin] RBAC: Adding query filter params (role.id=1d3b684bd8714b5792ebd7695c02a9ab, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=c48d41dcf9b344a8b2fec77f4dd75fcb) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:27.656670 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916431b-ff9a-4aaf-8ee2-dfb657fe1690 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:27.658307 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916431b-ff9a-4aaf-8ee2-dfb657fe1690 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:27.665019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 528/1047] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:57:27 2022] GET /identity/v3/role_assignments?user.id=c48d41dcf9b344a8b2fec77f4dd75fcb&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=1d3b684bd8714b5792ebd7695c02a9ab => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:27.670188 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5afaae-77fe-4bd4-986c-523790988ab3 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:27.805718 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5afaae-77fe-4bd4-986c-523790988ab3 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:27.853756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096ff6f-1d40-4a73-961c-9047db5994e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:27.854875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096ff6f-1d40-4a73-961c-9047db5994e2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:27.855201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096ff6f-1d40-4a73-961c-9047db5994e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:27.855507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096ff6f-1d40-4a73-961c-9047db5994e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:27.868322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7096ff6f-1d40-4a73-961c-9047db5994e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-7096ff6f-1d40-4a73-961c-9047db5994e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:57:27.876732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:27.877751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:27.878589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:27.878589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:27.878589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:27.878589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:27.878589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:57:27.878589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:57:27.878589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:57:27.878589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:57:27.878589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:57:27.878589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:57:27.878589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:57:27.878589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:27.878589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 520/1048] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:57:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:27.883901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916431b-ff9a-4aaf-8ee2-dfb657fe1690 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:27.999061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916431b-ff9a-4aaf-8ee2-dfb657fe1690 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:28.037013 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c44078-3d7e-45fb-873d-53a333a725d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:28.038069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c44078-3d7e-45fb-873d-53a333a725d5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:28.038534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c44078-3d7e-45fb-873d-53a333a725d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:28.038961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c44078-3d7e-45fb-873d-53a333a725d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:28.040014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c44078-3d7e-45fb-873d-53a333a725d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:28.040121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c44078-3d7e-45fb-873d-53a333a725d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:28.041426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c44078-3d7e-45fb-873d-53a333a725d5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:28.051779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 529/1049] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:57:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:28.058165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096ff6f-1d40-4a73-961c-9047db5994e2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:28.079454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:28.182897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096ff6f-1d40-4a73-961c-9047db5994e2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:28.223713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fb362-b7b3-4d79-b7b9-3060347e9803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:28.224930 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fb362-b7b3-4d79-b7b9-3060347e9803 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:28.225273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fb362-b7b3-4d79-b7b9-3060347e9803 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:28.225714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fb362-b7b3-4d79-b7b9-3060347e9803 None admin] PATH_INFO: `/v3/users/88c1f8c211b5489b9777a47c54db5aea` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:28.244929 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791fb362-b7b3-4d79-b7b9-3060347e9803 None admin] RBAC: Authorizing `identity:get_user(user_id=88c1f8c211b5489b9777a47c54db5aea)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:28.246411 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791fb362-b7b3-4d79-b7b9-3060347e9803 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:28.252839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 521/1050] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:57:28 2022] GET /identity/v3/users/88c1f8c211b5489b9777a47c54db5aea => generated 285 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:28.256604 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c44078-3d7e-45fb-873d-53a333a725d5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:28.299174 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:28.370727 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c44078-3d7e-45fb-873d-53a333a725d5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:28.401221 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0e4b6-e66c-484c-89a7-42a11de64c94 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:28.401549 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0e4b6-e66c-484c-89a7-42a11de64c94 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:28.401786 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0e4b6-e66c-484c-89a7-42a11de64c94 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:28.403297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 522/1051] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:57:28.410558 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefdc8a-1d6c-4585-ba82-6a72e1cea987 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:28.410758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8d6b4-2b7c-4480-bc9b-093d04e6cd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:28.410859 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefdc8a-1d6c-4585-ba82-6a72e1cea987 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:28.411298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8d6b4-2b7c-4480-bc9b-093d04e6cd45 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:28.411586 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8d6b4-2b7c-4480-bc9b-093d04e6cd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:28.411864 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8d6b4-2b7c-4480-bc9b-093d04e6cd45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:28.411975 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefdc8a-1d6c-4585-ba82-6a72e1cea987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:28.434325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2eefdc8a-1d6c-4585-ba82-6a72e1cea987 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:28.435511 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2eefdc8a-1d6c-4585-ba82-6a72e1cea987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:28.488947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8d6b4-2b7c-4480-bc9b-093d04e6cd45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:28.490422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8d6b4-2b7c-4480-bc9b-093d04e6cd45 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:28.530404 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:28.532435 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 523/1052] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:28.546388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefdc8a-1d6c-4585-ba82-6a72e1cea987 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:28.549491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:28.616353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 530/1053] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:57:28 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:57:28.623985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8d6b4-2b7c-4480-bc9b-093d04e6cd45 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:28.673098 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefdc8a-1d6c-4585-ba82-6a72e1cea987 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:28.718293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61377e18-6e0b-4e6f-bc34-34b290916b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:28.719128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61377e18-6e0b-4e6f-bc34-34b290916b0f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:28.719402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61377e18-6e0b-4e6f-bc34-34b290916b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:28.719677 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61377e18-6e0b-4e6f-bc34-34b290916b0f None admin] PATH_INFO: `/v3/services/image` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:28.720592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61377e18-6e0b-4e6f-bc34-34b290916b0f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:28.721751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61377e18-6e0b-4e6f-bc34-34b290916b0f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-61377e18-6e0b-4e6f-bc34-34b290916b0f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:28.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:28.737106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:28.738071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:28.738071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:28.738071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:28.738071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:28.738071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:28.738071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:28.738071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:28.738071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:28.738071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:28.738071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:28.738071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:57:28.738071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:57:28.738071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:57:28.738071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:28.738071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 524/1054] 10.222.0.13 () {56 vars in 1105 bytes} [Thu Jul 21 08:57:28 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:28.741602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61377e18-6e0b-4e6f-bc34-34b290916b0f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:28.755144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8d6b4-2b7c-4480-bc9b-093d04e6cd45 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:28.806367 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210065d-eeec-4fc5-962c-307cbb2dcbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:28.807166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210065d-eeec-4fc5-962c-307cbb2dcbed None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:28.807449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210065d-eeec-4fc5-962c-307cbb2dcbed None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:28.807725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210065d-eeec-4fc5-962c-307cbb2dcbed None admin] PATH_INFO: `/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:28.809091 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210065d-eeec-4fc5-962c-307cbb2dcbed None admin] RBAC: Authorizing `identity:get_project(project_id=4dc8f87e239a4cbfacfcfc057f00bd7d)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:28.810565 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210065d-eeec-4fc5-962c-307cbb2dcbed None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:28.813463 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 531/1055] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:57:28 2022] GET /identity/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d => generated 311 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:28.819214 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210065d-eeec-4fc5-962c-307cbb2dcbed None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:28.822281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:28.826090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:28.871602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61377e18-6e0b-4e6f-bc34-34b290916b0f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:28.913381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15121e6c-a441-4eb1-97bb-6831a4268b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:28.914060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15121e6c-a441-4eb1-97bb-6831a4268b4d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:28.914344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15121e6c-a441-4eb1-97bb-6831a4268b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:28.914623 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15121e6c-a441-4eb1-97bb-6831a4268b4d None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:28.915333 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15121e6c-a441-4eb1-97bb-6831a4268b4d None admin] RBAC: Adding query filter params (name=image) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:28.915644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15121e6c-a441-4eb1-97bb-6831a4268b4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:28.917074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15121e6c-a441-4eb1-97bb-6831a4268b4d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:28.931989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 525/1056] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:28.933419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15121e6c-a441-4eb1-97bb-6831a4268b4d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:28.946836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210065d-eeec-4fc5-962c-307cbb2dcbed None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:28.988200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d933a0-a12a-4cef-a2da-00b055a844e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:28.988862 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d933a0-a12a-4cef-a2da-00b055a844e9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:28.989141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d933a0-a12a-4cef-a2da-00b055a844e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:28.989418 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d933a0-a12a-4cef-a2da-00b055a844e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:28.990239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d933a0-a12a-4cef-a2da-00b055a844e9 None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=4dc8f87e239a4cbfacfcfc057f00bd7d, user.id=88c1f8c211b5489b9777a47c54db5aea) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:28.990548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d933a0-a12a-4cef-a2da-00b055a844e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:28.992013 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d933a0-a12a-4cef-a2da-00b055a844e9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:29.011944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 532/1057] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:57:28 2022] GET /identity/v3/role_assignments?user.id=88c1f8c211b5489b9777a47c54db5aea&scope.project.id=4dc8f87e239a4cbfacfcfc057f00bd7d&role.id=000e942517f2469ba36d2e65978bc314 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:29.064526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15121e6c-a441-4eb1-97bb-6831a4268b4d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:29.113555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:29.124204 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17656b7-3e48-498d-a249-c8dd02dc3e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:29.124907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17656b7-3e48-498d-a249-c8dd02dc3e64 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:29.125854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17656b7-3e48-498d-a249-c8dd02dc3e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:29.126868 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17656b7-3e48-498d-a249-c8dd02dc3e64 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:29.127635 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17656b7-3e48-498d-a249-c8dd02dc3e64 None admin] RBAC: Adding query filter params (type=image) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:29.127970 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17656b7-3e48-498d-a249-c8dd02dc3e64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:29.129438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17656b7-3e48-498d-a249-c8dd02dc3e64 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:29.135610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 526/1058] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:29.142111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d933a0-a12a-4cef-a2da-00b055a844e9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:29.227143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:29.271116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d933a0-a12a-4cef-a2da-00b055a844e9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:29.312376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5361c-b58d-41a1-9c12-a0df90cf07eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:29.312376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5361c-b58d-41a1-9c12-a0df90cf07eb None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:29.312674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5361c-b58d-41a1-9c12-a0df90cf07eb None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:29.312935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5361c-b58d-41a1-9c12-a0df90cf07eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:29.314018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5361c-b58d-41a1-9c12-a0df90cf07eb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:29.315177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5361c-b58d-41a1-9c12-a0df90cf07eb None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:29.339532 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 533/1059] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jul 21 08:57:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:57:29.763033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3fcc-2c90-44ae-8525-328e24a0d777 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:29.763384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3fcc-2c90-44ae-8525-328e24a0d777 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:29.763591 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3fcc-2c90-44ae-8525-328e24a0d777 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:29.765183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 527/1060] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:29.772660 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09b8c2-0de6-4674-ac6f-ee90905f406f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:29.772993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09b8c2-0de6-4674-ac6f-ee90905f406f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:29.773275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09b8c2-0de6-4674-ac6f-ee90905f406f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:29.791703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-da09b8c2-0de6-4674-ac6f-ee90905f406f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:29.792920 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-da09b8c2-0de6-4674-ac6f-ee90905f406f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:29.881465 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 534/1061] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:29.904238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b3fcc-2c90-44ae-8525-328e24a0d777 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:30.020263 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b3fcc-2c90-44ae-8525-328e24a0d777 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:30.060190 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6a914-8245-4780-9e58-51601513710d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:30.061007 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6a914-8245-4780-9e58-51601513710d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:30.061292 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6a914-8245-4780-9e58-51601513710d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:30.061568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6a914-8245-4780-9e58-51601513710d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:30.073530 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6a914-8245-4780-9e58-51601513710d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-59d6a914-8245-4780-9e58-51601513710d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:57:30.082599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:30.083609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:30.085402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:30.085402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:30.085402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:30.085402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:30.085402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:57:30.085402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:57:30.085402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:57:30.085402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:57:30.085402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:57:30.085402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:57:30.085402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:57:30.085402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:30.088160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 528/1062] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jul 21 08:57:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:30.090058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09b8c2-0de6-4674-ac6f-ee90905f406f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:30.155742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:30.221234 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09b8c2-0de6-4674-ac6f-ee90905f406f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:30.259509 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd97d34-d6d8-4a02-bdfb-8c8eb6a0e0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:30.260200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97d34-d6d8-4a02-bdfb-8c8eb6a0e0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:30.260484 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97d34-d6d8-4a02-bdfb-8c8eb6a0e0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:30.260758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97d34-d6d8-4a02-bdfb-8c8eb6a0e0e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:30.261504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd97d34-d6d8-4a02-bdfb-8c8eb6a0e0e8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:30.261806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd97d34-d6d8-4a02-bdfb-8c8eb6a0e0e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:30.263175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd97d34-d6d8-4a02-bdfb-8c8eb6a0e0e8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:30.278309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 535/1063] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jul 21 08:57:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:30.281006 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6a914-8245-4780-9e58-51601513710d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:30.345762 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:30.396404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6a914-8245-4780-9e58-51601513710d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:30.440570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185934ec-a2dc-4f6e-83ea-e308d7416f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:30.441426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185934ec-a2dc-4f6e-83ea-e308d7416f22 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:30.441709 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185934ec-a2dc-4f6e-83ea-e308d7416f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:30.441984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185934ec-a2dc-4f6e-83ea-e308d7416f22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:30.454221 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185934ec-a2dc-4f6e-83ea-e308d7416f22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:30.455560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185934ec-a2dc-4f6e-83ea-e308d7416f22 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:30.473360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-185934ec-a2dc-4f6e-83ea-e308d7416f22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:30.473360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:30.473360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:30.473360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:30.473360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:30.473360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:30.473360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:30.473360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:30.473360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:30.473360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:30.473360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:30.473360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:30.473360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:30.473360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:30.473360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:30.474388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:30.476245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:30.476245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:30.476245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:30.476245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:30.476739 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 529/1064] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:30.480682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd97d34-d6d8-4a02-bdfb-8c8eb6a0e0e8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:30.609290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd97d34-d6d8-4a02-bdfb-8c8eb6a0e0e8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:30.619859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:30.651630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0592efb-2fea-4311-b362-a49b7354a300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:30.652689 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0592efb-2fea-4311-b362-a49b7354a300 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:30.653121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0592efb-2fea-4311-b362-a49b7354a300 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:30.653559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0592efb-2fea-4311-b362-a49b7354a300 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:30.654548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0592efb-2fea-4311-b362-a49b7354a300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:30.655017 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0592efb-2fea-4311-b362-a49b7354a300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:30.657161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0592efb-2fea-4311-b362-a49b7354a300 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:30.675348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 536/1065] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:30.676630 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185934ec-a2dc-4f6e-83ea-e308d7416f22 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:30.789522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185934ec-a2dc-4f6e-83ea-e308d7416f22 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:30.796539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:30.828571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d06faa-61a6-47fd-869e-71aee02a15cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:30.829358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d06faa-61a6-47fd-869e-71aee02a15cb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:30.829635 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d06faa-61a6-47fd-869e-71aee02a15cb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:30.829908 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d06faa-61a6-47fd-869e-71aee02a15cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:30.837685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d06faa-61a6-47fd-869e-71aee02a15cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:30.839205 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d06faa-61a6-47fd-869e-71aee02a15cb None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-36d06faa-61a6-47fd-869e-71aee02a15cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:57:30.854901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:30.855907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:30.863319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:30.863319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:30.863319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:57:30.863319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:57:30.863319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:57:30.863319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:57:30.863319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:57:30.863319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:57:30.863319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:57:30.863319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:30.863319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 530/1066] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jul 21 08:57:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:30.863954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0592efb-2fea-4311-b362-a49b7354a300 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:30.970601 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0592efb-2fea-4311-b362-a49b7354a300 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:31.006573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad1d10-f4a8-45bd-831e-3747ba5df410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:31.007619 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad1d10-f4a8-45bd-831e-3747ba5df410 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:31.008158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad1d10-f4a8-45bd-831e-3747ba5df410 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:31.008653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad1d10-f4a8-45bd-831e-3747ba5df410 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:31.009694 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad1d10-f4a8-45bd-831e-3747ba5df410 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:31.010201 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad1d10-f4a8-45bd-831e-3747ba5df410 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:31.011839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ad1d10-f4a8-45bd-831e-3747ba5df410 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:31.036004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 537/1067] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jul 21 08:57:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:31.037384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d06faa-61a6-47fd-869e-71aee02a15cb None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:31.088686 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb1314-8828-4f4a-8727-8febbb959edb None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:31.088979 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb1314-8828-4f4a-8727-8febbb959edb None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:31.089258 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb1314-8828-4f4a-8727-8febbb959edb None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:31.090768 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 538/1068] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:31.096307 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af12532-5081-48d8-a918-0ed6028b604a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:31.096605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af12532-5081-48d8-a918-0ed6028b604a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:31.096884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af12532-5081-48d8-a918-0ed6028b604a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:31.114472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7af12532-5081-48d8-a918-0ed6028b604a None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:31.115606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7af12532-5081-48d8-a918-0ed6028b604a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:31.141464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:31.151357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d06faa-61a6-47fd-869e-71aee02a15cb None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:31.192566 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 539/1069] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:31.202901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c51c41-9dd3-493c-a685-082ff6dfe099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:31.203760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c51c41-9dd3-493c-a685-082ff6dfe099 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:31.204158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c51c41-9dd3-493c-a685-082ff6dfe099 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:31.204580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c51c41-9dd3-493c-a685-082ff6dfe099 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:31.215234 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af12532-5081-48d8-a918-0ed6028b604a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:31.218133 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c51c41-9dd3-493c-a685-082ff6dfe099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:31.219819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c51c41-9dd3-493c-a685-082ff6dfe099 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:31.240150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-e7c51c41-9dd3-493c-a685-082ff6dfe099 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:31.240150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:31.240150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:31.240150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:31.240150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:31.240150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:31.240150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:31.240150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:31.240150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:31.240150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:31.240150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:31.240150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:31.240150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:31.240150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:31.240150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:31.241153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:31.242051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:31.242051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:31.242051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:31.242051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:31.242051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 531/1070] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:31.247094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c51c41-9dd3-493c-a685-082ff6dfe099 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:31.268119 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:31.363270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c51c41-9dd3-493c-a685-082ff6dfe099 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:31.366023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af12532-5081-48d8-a918-0ed6028b604a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:31.406299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db222b40-e546-4c66-9a6a-6015c8e1b1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:31.407842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db222b40-e546-4c66-9a6a-6015c8e1b1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:31.408325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23eacd2-e43c-46d7-9a1d-6f52f6772932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:31.408998 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23eacd2-e43c-46d7-9a1d-6f52f6772932 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:31.409286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23eacd2-e43c-46d7-9a1d-6f52f6772932 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:31.409451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db222b40-e546-4c66-9a6a-6015c8e1b1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:31.409600 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23eacd2-e43c-46d7-9a1d-6f52f6772932 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:31.409689 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db222b40-e546-4c66-9a6a-6015c8e1b1d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:31.410303 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23eacd2-e43c-46d7-9a1d-6f52f6772932 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:31.410614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23eacd2-e43c-46d7-9a1d-6f52f6772932 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:31.413194 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23eacd2-e43c-46d7-9a1d-6f52f6772932 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:31.421339 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db222b40-e546-4c66-9a6a-6015c8e1b1d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:57:31.427946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 532/1071] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-db222b40-e546-4c66-9a6a-6015c8e1b1d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:57:31.430397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:31.431299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:31.438560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:31.438560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:31.438560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:31.438560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:31.438560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:57:31.438560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:57:31.438560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:57:31.438560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:57:31.438560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:57:31.438560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:57:31.438560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:57:31.438560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:31.438560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 540/1072] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:57:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:31.439369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23eacd2-e43c-46d7-9a1d-6f52f6772932 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:31.439458 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db222b40-e546-4c66-9a6a-6015c8e1b1d4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:31.460595 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:31.533066 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:31.555405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db222b40-e546-4c66-9a6a-6015c8e1b1d4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:31.561914 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23eacd2-e43c-46d7-9a1d-6f52f6772932 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:31.595345 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ad375-d31e-457b-ad8a-4eeb9cad3638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:31.596091 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ad375-d31e-457b-ad8a-4eeb9cad3638 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:31.596567 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ad375-d31e-457b-ad8a-4eeb9cad3638 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:31.597069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ad375-d31e-457b-ad8a-4eeb9cad3638 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:31.598202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ad375-d31e-457b-ad8a-4eeb9cad3638 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:31.598512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ad375-d31e-457b-ad8a-4eeb9cad3638 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:31.600581 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ad375-d31e-457b-ad8a-4eeb9cad3638 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:31.601807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac101af8-7341-4b29-8b59-36e58e489a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:31.602593 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac101af8-7341-4b29-8b59-36e58e489a4a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:31.602896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac101af8-7341-4b29-8b59-36e58e489a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:31.603141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac101af8-7341-4b29-8b59-36e58e489a4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:31.610830 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac101af8-7341-4b29-8b59-36e58e489a4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:31.612849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac101af8-7341-4b29-8b59-36e58e489a4a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:31.613731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 541/1073] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:57:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:31.615383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a6ec2-cc27-4760-b686-0d9479d0a294 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:31.615676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a6ec2-cc27-4760-b686-0d9479d0a294 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:31.615974 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a6ec2-cc27-4760-b686-0d9479d0a294 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:31.617306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 542/1074] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:31.622573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528ccd0-ac5a-46bc-8d5d-bc7db13c20d7 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:31.622862 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528ccd0-ac5a-46bc-8d5d-bc7db13c20d7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:31.623141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528ccd0-ac5a-46bc-8d5d-bc7db13c20d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:31.634016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-ac101af8-7341-4b29-8b59-36e58e489a4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:31.634016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:31.634016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:31.634016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:31.634016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:31.634016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:31.634016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:31.634016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:31.634016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:31.634016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:31.634016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:31.634016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:31.634016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:57:31.634016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:57:31.634016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:31.634983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:31.636002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:57:31.636002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:57:31.636002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:57:31.636002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:57:31.636002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:57:31.636002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:57:31.636002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:31.636002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:31.636002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 533/1075] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:57:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:31.636846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac101af8-7341-4b29-8b59-36e58e489a4a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:31.645591 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7528ccd0-ac5a-46bc-8d5d-bc7db13c20d7 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:31.646846 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7528ccd0-ac5a-46bc-8d5d-bc7db13c20d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:31.738831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 543/1076] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:31.738831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528ccd0-ac5a-46bc-8d5d-bc7db13c20d7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:31.750613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:31.768410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac101af8-7341-4b29-8b59-36e58e489a4a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:31.808342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629de6ef-5463-4176-9dcd-8adc92410a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:31.809217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629de6ef-5463-4176-9dcd-8adc92410a1d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:31.809495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629de6ef-5463-4176-9dcd-8adc92410a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:31.809764 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629de6ef-5463-4176-9dcd-8adc92410a1d None admin] PATH_INFO: `/v3/users/88c1f8c211b5489b9777a47c54db5aea` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:31.812428 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629de6ef-5463-4176-9dcd-8adc92410a1d None admin] RBAC: Authorizing `identity:get_user(user_id=88c1f8c211b5489b9777a47c54db5aea)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:31.814080 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629de6ef-5463-4176-9dcd-8adc92410a1d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:31.816070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 534/1077] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:57:31 2022] GET /identity/v3/users/88c1f8c211b5489b9777a47c54db5aea => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:31.816674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629de6ef-5463-4176-9dcd-8adc92410a1d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:31.827177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:31.855569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528ccd0-ac5a-46bc-8d5d-bc7db13c20d7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:31.895252 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d90893-34c6-4b20-9472-e5e55e06b02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:31.895928 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d90893-34c6-4b20-9472-e5e55e06b02b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:31.896227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d90893-34c6-4b20-9472-e5e55e06b02b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:31.896505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d90893-34c6-4b20-9472-e5e55e06b02b None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:31.897306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d90893-34c6-4b20-9472-e5e55e06b02b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:31.897613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d90893-34c6-4b20-9472-e5e55e06b02b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:31.899068 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d90893-34c6-4b20-9472-e5e55e06b02b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:31.913304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 544/1078] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:57:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:31.913523 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d90893-34c6-4b20-9472-e5e55e06b02b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:31.949695 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629de6ef-5463-4176-9dcd-8adc92410a1d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:31.993396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:31.998636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af717e-67ed-4bd9-b596-f75aa798f0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:31.999296 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af717e-67ed-4bd9-b596-f75aa798f0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:31.999576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af717e-67ed-4bd9-b596-f75aa798f0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:31.999850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af717e-67ed-4bd9-b596-f75aa798f0a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:32.029682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d90893-34c6-4b20-9472-e5e55e06b02b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:32.073606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9b349-e951-489d-a5af-77f5968bf9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:32.074479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9b349-e951-489d-a5af-77f5968bf9cf None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:32.074760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9b349-e951-489d-a5af-77f5968bf9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:32.075031 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9b349-e951-489d-a5af-77f5968bf9cf None admin] PATH_INFO: `/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:32.076247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9b349-e951-489d-a5af-77f5968bf9cf None admin] RBAC: Authorizing `identity:get_project(project_id=4dc8f87e239a4cbfacfcfc057f00bd7d)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:32.077702 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9b349-e951-489d-a5af-77f5968bf9cf None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:32.079903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 545/1079] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:57:31 2022] GET /identity/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:32.080673 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9b349-e951-489d-a5af-77f5968bf9cf None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:32.089168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af717e-67ed-4bd9-b596-f75aa798f0a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:32.090644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af717e-67ed-4bd9-b596-f75aa798f0a9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:32.099258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:32.185499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9b349-e951-489d-a5af-77f5968bf9cf None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:32.201729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 535/1080] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:57:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:57:32.202562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af717e-67ed-4bd9-b596-f75aa798f0a9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:32.225776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8ef36-64dc-4310-90c8-5f4b6e70eeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:32.226663 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8ef36-64dc-4310-90c8-5f4b6e70eeea None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:32.226941 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8ef36-64dc-4310-90c8-5f4b6e70eeea None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:32.227208 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8ef36-64dc-4310-90c8-5f4b6e70eeea None admin] PATH_INFO: `/v3/projects/7fe0f489cbce43859f050c3270e00709/users/c48d41dcf9b344a8b2fec77f4dd75fcb/roles/1d3b684bd8714b5792ebd7695c02a9ab` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:32.229457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8ef36-64dc-4310-90c8-5f4b6e70eeea None admin] RBAC: Authorizing `identity:create_grant(project_id=7fe0f489cbce43859f050c3270e00709, user_id=c48d41dcf9b344a8b2fec77f4dd75fcb, role_id=1d3b684bd8714b5792ebd7695c02a9ab)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:32.230894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8ef36-64dc-4310-90c8-5f4b6e70eeea None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:32.239348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:32.245463 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 546/1081] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:57:32 2022] PUT /identity/v3/projects/7fe0f489cbce43859f050c3270e00709/users/c48d41dcf9b344a8b2fec77f4dd75fcb/roles/1d3b684bd8714b5792ebd7695c02a9ab => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:57:32.246721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8ef36-64dc-4310-90c8-5f4b6e70eeea None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:32.318219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af717e-67ed-4bd9-b596-f75aa798f0a9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:32.328003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:32.362103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c59ef-deac-49d7-96f9-38495e67dba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:32.363025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c59ef-deac-49d7-96f9-38495e67dba0 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:32.363308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c59ef-deac-49d7-96f9-38495e67dba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:32.363839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c59ef-deac-49d7-96f9-38495e67dba0 None admin] PATH_INFO: `/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d/users/88c1f8c211b5489b9777a47c54db5aea/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:32.365680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c59ef-deac-49d7-96f9-38495e67dba0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4dc8f87e239a4cbfacfcfc057f00bd7d, user_id=88c1f8c211b5489b9777a47c54db5aea, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:32.367172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c59ef-deac-49d7-96f9-38495e67dba0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:32.370195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8ef36-64dc-4310-90c8-5f4b6e70eeea None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:32.384515 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 536/1082] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:57:32 2022] PUT /identity/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d/users/88c1f8c211b5489b9777a47c54db5aea/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:57:32.417789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23572a7d-271e-4151-b3b2-fbec7396616d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:32.419403 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23572a7d-271e-4151-b3b2-fbec7396616d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:32.419403 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23572a7d-271e-4151-b3b2-fbec7396616d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:32.419403 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23572a7d-271e-4151-b3b2-fbec7396616d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:32.434649 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23572a7d-271e-4151-b3b2-fbec7396616d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:32.436260 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23572a7d-271e-4151-b3b2-fbec7396616d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:32.456414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-23572a7d-271e-4151-b3b2-fbec7396616d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:32.456414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:32.456414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:32.456414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:32.456414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:32.456414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:32.456414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:32.456414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:32.456414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:32.456414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:32.456414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:32.456414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:32.456414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:32.456414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:32.456414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:32.457396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:32.458751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:32.458751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:32.458751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:32.458751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:32.458751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 547/1083] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:32.464165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c59ef-deac-49d7-96f9-38495e67dba0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:32.579399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c59ef-deac-49d7-96f9-38495e67dba0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:32.617238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453d00c0-dd67-4509-8131-8c186fb71b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:32.617943 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d00c0-dd67-4509-8131-8c186fb71b8e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:32.618228 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d00c0-dd67-4509-8131-8c186fb71b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:32.618505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453d00c0-dd67-4509-8131-8c186fb71b8e None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:32.619257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d00c0-dd67-4509-8131-8c186fb71b8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:32.619562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d00c0-dd67-4509-8131-8c186fb71b8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:32.621055 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453d00c0-dd67-4509-8131-8c186fb71b8e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:32.638224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 537/1084] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:34.307042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace428f4-337b-4015-903b-7d4e957fef9b None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:34.307581 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace428f4-337b-4015-903b-7d4e957fef9b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:34.307581 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace428f4-337b-4015-903b-7d4e957fef9b None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:34.309013 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 548/1085] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:34.314384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cea01a-7170-4d9f-9a7b-4f54b7668945 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:34.314826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cea01a-7170-4d9f-9a7b-4f54b7668945 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:34.314914 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cea01a-7170-4d9f-9a7b-4f54b7668945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:34.332293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-47cea01a-7170-4d9f-9a7b-4f54b7668945 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:34.333491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-47cea01a-7170-4d9f-9a7b-4f54b7668945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:34.354612 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:34.403173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 538/1086] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:34.426397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace428f4-337b-4015-903b-7d4e957fef9b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:34.480165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:34.504957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec54d69-09d7-4572-83a1-e62fd9392b00 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:34.505252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec54d69-09d7-4572-83a1-e62fd9392b00 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:34.505531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec54d69-09d7-4572-83a1-e62fd9392b00 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:34.506909 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 539/1087] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:34.512001 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c519d36-166b-4f69-b2ef-0548deef54d1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:34.512291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c519d36-166b-4f69-b2ef-0548deef54d1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:34.512572 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c519d36-166b-4f69-b2ef-0548deef54d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:34.530558 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5c519d36-166b-4f69-b2ef-0548deef54d1 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:34.531709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace428f4-337b-4015-903b-7d4e957fef9b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:34.532446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5c519d36-166b-4f69-b2ef-0548deef54d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:34.569194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873a002-311f-4348-b994-d6d07b767a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:34.570028 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873a002-311f-4348-b994-d6d07b767a03 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:34.570221 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873a002-311f-4348-b994-d6d07b767a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:34.570493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873a002-311f-4348-b994-d6d07b767a03 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:34.581969 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873a002-311f-4348-b994-d6d07b767a03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-4873a002-311f-4348-b994-d6d07b767a03 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:57:34.590190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:34.592059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:34.593662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:34.593662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:34.593662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:34.593662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:34.593662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:57:34.593662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:57:34.593662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:57:34.593662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:57:34.593662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:57:34.593662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:57:34.593662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:57:34.593662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:34.596164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 549/1088] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jul 21 08:57:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:34.597919 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873a002-311f-4348-b994-d6d07b767a03 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:34.611969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 540/1089] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:34.636000 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c519d36-166b-4f69-b2ef-0548deef54d1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:34.731251 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:34.736799 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873a002-311f-4348-b994-d6d07b767a03 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:34.748842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c519d36-166b-4f69-b2ef-0548deef54d1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:34.749524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:34.785909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea18bd-a5f8-44fa-9fa9-19bbd1ed7e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:34.786571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea18bd-a5f8-44fa-9fa9-19bbd1ed7e79 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:34.786852 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea18bd-a5f8-44fa-9fa9-19bbd1ed7e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:34.787126 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea18bd-a5f8-44fa-9fa9-19bbd1ed7e79 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:34.787870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea18bd-a5f8-44fa-9fa9-19bbd1ed7e79 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:34.788015 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096594e5-d75e-4f99-b98c-66df62d95e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:34.788202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea18bd-a5f8-44fa-9fa9-19bbd1ed7e79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:34.788704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096594e5-d75e-4f99-b98c-66df62d95e6e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:34.788989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096594e5-d75e-4f99-b98c-66df62d95e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:34.789296 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096594e5-d75e-4f99-b98c-66df62d95e6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:34.789565 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea18bd-a5f8-44fa-9fa9-19bbd1ed7e79 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:34.797977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 550/1090] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jul 21 08:57:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:34.800212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096594e5-d75e-4f99-b98c-66df62d95e6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:57:34.803850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea18bd-a5f8-44fa-9fa9-19bbd1ed7e79 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-096594e5-d75e-4f99-b98c-66df62d95e6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:57:34.807804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:34.808881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:34.809729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:34.809729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:34.809729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:34.809729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:34.809729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:57:34.809729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:57:34.809729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:57:34.809729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:57:34.809729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:57:34.809729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:57:34.809729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:57:34.809729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:34.809729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 541/1091] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:57:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:34.815446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096594e5-d75e-4f99-b98c-66df62d95e6e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:34.904315 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea18bd-a5f8-44fa-9fa9-19bbd1ed7e79 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:34.917315 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096594e5-d75e-4f99-b98c-66df62d95e6e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:34.939815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421c4b0-1325-44a3-bd02-32a72d1b19a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:34.940670 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421c4b0-1325-44a3-bd02-32a72d1b19a0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:34.940957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421c4b0-1325-44a3-bd02-32a72d1b19a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:34.941234 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421c4b0-1325-44a3-bd02-32a72d1b19a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:34.952625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421c4b0-1325-44a3-bd02-32a72d1b19a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:34.954764 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421c4b0-1325-44a3-bd02-32a72d1b19a0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:34.955091 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7fa03-0e65-4a58-ae76-9188cc71a2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:34.955767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7fa03-0e65-4a58-ae76-9188cc71a2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:34.956089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7fa03-0e65-4a58-ae76-9188cc71a2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:34.956375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7fa03-0e65-4a58-ae76-9188cc71a2a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:34.957127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7fa03-0e65-4a58-ae76-9188cc71a2a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:34.957441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7fa03-0e65-4a58-ae76-9188cc71a2a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:34.958826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7fa03-0e65-4a58-ae76-9188cc71a2a6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:34.967352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 542/1092] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:57:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:34.974046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7fa03-0e65-4a58-ae76-9188cc71a2a6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:34.977039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-c421c4b0-1325-44a3-bd02-32a72d1b19a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:34.977039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:34.977039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:34.977039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:34.977039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:34.977039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:34.977039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:34.977039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:34.977039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:34.977039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:34.977039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:34.977039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:34.977039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:34.977039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:34.977039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:34.977963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:34.983710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:34.983710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:34.983710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:34.983710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:34.983710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 551/1093] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:34.984107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421c4b0-1325-44a3-bd02-32a72d1b19a0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:34.989391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:35.012873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:35.089413 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421c4b0-1325-44a3-bd02-32a72d1b19a0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:35.089762 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7fa03-0e65-4a58-ae76-9188cc71a2a6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:35.123398 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb638e-70f3-4b2d-b3f6-b24122ea44bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:35.124095 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb638e-70f3-4b2d-b3f6-b24122ea44bb None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:35.124379 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb638e-70f3-4b2d-b3f6-b24122ea44bb None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:35.124650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb638e-70f3-4b2d-b3f6-b24122ea44bb None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:35.125941 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb638e-70f3-4b2d-b3f6-b24122ea44bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:35.126208 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004567e-d2a4-4451-b4cd-cfefba6b078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:35.126327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb638e-70f3-4b2d-b3f6-b24122ea44bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:35.127649 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb638e-70f3-4b2d-b3f6-b24122ea44bb None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:35.127769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004567e-d2a4-4451-b4cd-cfefba6b078f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:35.127964 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004567e-d2a4-4451-b4cd-cfefba6b078f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:35.128249 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2004567e-d2a4-4451-b4cd-cfefba6b078f None admin] PATH_INFO: `/v3/users/88c1f8c211b5489b9777a47c54db5aea` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:35.129620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004567e-d2a4-4451-b4cd-cfefba6b078f None admin] RBAC: Authorizing `identity:get_user(user_id=88c1f8c211b5489b9777a47c54db5aea)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:35.131105 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2004567e-d2a4-4451-b4cd-cfefba6b078f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:35.132899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 543/1094] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:57:34 2022] GET /identity/v3/users/88c1f8c211b5489b9777a47c54db5aea => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:35.138913 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004567e-d2a4-4451-b4cd-cfefba6b078f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:35.141886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 552/1095] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:35.147800 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb638e-70f3-4b2d-b3f6-b24122ea44bb None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:35.228630 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:35.241027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:35.254920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2004567e-d2a4-4451-b4cd-cfefba6b078f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:35.260136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb638e-70f3-4b2d-b3f6-b24122ea44bb None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:35.295504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f4d16-58a2-4217-a5a6-191124f3944a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:35.296339 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f4d16-58a2-4217-a5a6-191124f3944a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:35.296711 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f4d16-58a2-4217-a5a6-191124f3944a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:35.297093 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f4d16-58a2-4217-a5a6-191124f3944a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:35.298913 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a02c1-8732-4fa0-b450-ff6e5f3db2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:35.299718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a02c1-8732-4fa0-b450-ff6e5f3db2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:35.300031 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a02c1-8732-4fa0-b450-ff6e5f3db2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:35.300301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a02c1-8732-4fa0-b450-ff6e5f3db2b2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:35.308553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a02c1-8732-4fa0-b450-ff6e5f3db2b2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:35.310030 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a02c1-8732-4fa0-b450-ff6e5f3db2b2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-585a02c1-8732-4fa0-b450-ff6e5f3db2b2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:57:35.325386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:35.326867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:35.328556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:35.328556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:35.328556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:57:35.328556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:57:35.328556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:57:35.328556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:57:35.328556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:57:35.328556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:57:35.328556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:57:35.328556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:35.328556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 553/1096] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jul 21 08:57:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:57:35.332673 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a02c1-8732-4fa0-b450-ff6e5f3db2b2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:35.387743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f4d16-58a2-4217-a5a6-191124f3944a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:35.389253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f4d16-58a2-4217-a5a6-191124f3944a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:35.460983 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a02c1-8732-4fa0-b450-ff6e5f3db2b2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:35.481484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:35.503318 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91bf75-e7c8-4018-9d23-bfbd446be428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:35.504047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91bf75-e7c8-4018-9d23-bfbd446be428 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:35.504334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91bf75-e7c8-4018-9d23-bfbd446be428 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:35.504605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91bf75-e7c8-4018-9d23-bfbd446be428 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:35.505476 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91bf75-e7c8-4018-9d23-bfbd446be428 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:35.505781 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91bf75-e7c8-4018-9d23-bfbd446be428 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:35.507288 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91bf75-e7c8-4018-9d23-bfbd446be428 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:35.516733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 544/1097] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:57:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 08:57:35.523086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f4d16-58a2-4217-a5a6-191124f3944a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:35.531310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 554/1098] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jul 21 08:57:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:35.539084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91bf75-e7c8-4018-9d23-bfbd446be428 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:35.542776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:35.669934 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f4d16-58a2-4217-a5a6-191124f3944a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:35.675327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91bf75-e7c8-4018-9d23-bfbd446be428 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:35.731095 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98eaa03-0a4e-4b14-a022-d56d8ff70fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:35.731968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98eaa03-0a4e-4b14-a022-d56d8ff70fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:35.732210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98eaa03-0a4e-4b14-a022-d56d8ff70fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:35.732487 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98eaa03-0a4e-4b14-a022-d56d8ff70fd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:35.734227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c7863-1199-41fa-85ff-54883a4a8e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:35.737088 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7863-1199-41fa-85ff-54883a4a8e55 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:35.739160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7863-1199-41fa-85ff-54883a4a8e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:35.739653 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7863-1199-41fa-85ff-54883a4a8e55 None admin] PATH_INFO: `/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:35.741975 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c7863-1199-41fa-85ff-54883a4a8e55 None admin] RBAC: Authorizing `identity:get_project(project_id=4dc8f87e239a4cbfacfcfc057f00bd7d)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:35.743700 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c7863-1199-41fa-85ff-54883a4a8e55 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:35.746299 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 545/1099] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:57:35 2022] GET /identity/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d => generated 311 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:35.754946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c7863-1199-41fa-85ff-54883a4a8e55 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:35.805956 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:35.824904 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98eaa03-0a4e-4b14-a022-d56d8ff70fd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:35.826387 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98eaa03-0a4e-4b14-a022-d56d8ff70fd0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:35.828343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:35.869626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c7863-1199-41fa-85ff-54883a4a8e55 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:35.910360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9c5e4-0125-4a8e-ad33-2746ed1a058b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:35.911288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9c5e4-0125-4a8e-ad33-2746ed1a058b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:35.911593 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9c5e4-0125-4a8e-ad33-2746ed1a058b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:35.911832 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9c5e4-0125-4a8e-ad33-2746ed1a058b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:35.912705 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9c5e4-0125-4a8e-ad33-2746ed1a058b None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=4dc8f87e239a4cbfacfcfc057f00bd7d, user.id=88c1f8c211b5489b9777a47c54db5aea) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:35.913011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9c5e4-0125-4a8e-ad33-2746ed1a058b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:35.914456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9c5e4-0125-4a8e-ad33-2746ed1a058b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:35.920917 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 546/1100] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:57:35 2022] GET /identity/v3/role_assignments?user.id=88c1f8c211b5489b9777a47c54db5aea&scope.project.id=4dc8f87e239a4cbfacfcfc057f00bd7d&role.id=000e942517f2469ba36d2e65978bc314 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:35.938052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 555/1101] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:57:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 08:57:35.944434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9c5e4-0125-4a8e-ad33-2746ed1a058b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:36.074132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9c5e4-0125-4a8e-ad33-2746ed1a058b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:36.084299 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:36.112271 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822e6cf-cbab-45db-8f6e-64d6ff7f86aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:36.113072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822e6cf-cbab-45db-8f6e-64d6ff7f86aa None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:36.113361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822e6cf-cbab-45db-8f6e-64d6ff7f86aa None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:36.113644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822e6cf-cbab-45db-8f6e-64d6ff7f86aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:36.124870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822e6cf-cbab-45db-8f6e-64d6ff7f86aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:36.126916 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822e6cf-cbab-45db-8f6e-64d6ff7f86aa None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:36.145746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-8822e6cf-cbab-45db-8f6e-64d6ff7f86aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:36.145746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:36.145746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:36.145746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:36.145746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:36.145746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:36.145746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:36.145746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:36.145746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:36.145746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:36.145746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:36.145746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:36.145746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:57:36.145746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:57:36.145746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:36.146716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:36.147599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:57:36.147599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:57:36.147599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:57:36.147599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:36.147599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 547/1102] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:36.153112 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98eaa03-0a4e-4b14-a022-d56d8ff70fd0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:36.265668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98eaa03-0a4e-4b14-a022-d56d8ff70fd0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:36.267489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:36.303770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50f2f1-6583-4f57-8595-df63c86baaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:36.304465 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50f2f1-6583-4f57-8595-df63c86baaab None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:36.304745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50f2f1-6583-4f57-8595-df63c86baaab None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:36.305207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50f2f1-6583-4f57-8595-df63c86baaab None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:36.305945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50f2f1-6583-4f57-8595-df63c86baaab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:36.306250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50f2f1-6583-4f57-8595-df63c86baaab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:36.307650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50f2f1-6583-4f57-8595-df63c86baaab None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:36.320947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 556/1103] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:36.326885 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822e6cf-cbab-45db-8f6e-64d6ff7f86aa None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:36.447456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822e6cf-cbab-45db-8f6e-64d6ff7f86aa None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:36.482635 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cd373-42d0-4c0a-9dc6-8ebbbb35f78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:36.483429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cd373-42d0-4c0a-9dc6-8ebbbb35f78f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:36.483712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cd373-42d0-4c0a-9dc6-8ebbbb35f78f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:36.484020 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cd373-42d0-4c0a-9dc6-8ebbbb35f78f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:36.492086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cd373-42d0-4c0a-9dc6-8ebbbb35f78f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:36.493538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cd373-42d0-4c0a-9dc6-8ebbbb35f78f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:36.509046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-878cd373-42d0-4c0a-9dc6-8ebbbb35f78f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:36.509046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:36.509046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:36.509046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:36.509046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:36.509046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:36.509046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:36.509046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:36.509046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:36.509046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:36.509046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:36.509046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:36.509046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:57:36.509046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:57:36.509046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:36.509984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:36.510896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:57:36.510896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:57:36.510896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:57:36.510896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:57:36.510896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:57:36.510896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:57:36.510896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:57:36.510896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:57:36.510896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 548/1104] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:57:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:36.516868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50f2f1-6583-4f57-8595-df63c86baaab None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:36.629355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50f2f1-6583-4f57-8595-df63c86baaab None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:36.663511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60fd31-519e-454f-ac1d-dd8b0584a7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:36.664213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60fd31-519e-454f-ac1d-dd8b0584a7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:36.664501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60fd31-519e-454f-ac1d-dd8b0584a7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:36.664779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60fd31-519e-454f-ac1d-dd8b0584a7f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:36.665575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60fd31-519e-454f-ac1d-dd8b0584a7f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:36.665889 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60fd31-519e-454f-ac1d-dd8b0584a7f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:36.667342 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60fd31-519e-454f-ac1d-dd8b0584a7f6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:36.680954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 557/1105] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:57:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:36.687228 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cd373-42d0-4c0a-9dc6-8ebbbb35f78f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:36.710943 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:36.792260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cd373-42d0-4c0a-9dc6-8ebbbb35f78f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:36.826959 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a8b4f3-a758-4834-afaf-92fdd7525a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:36.827608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8b4f3-a758-4834-afaf-92fdd7525a83 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:36.827892 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8b4f3-a758-4834-afaf-92fdd7525a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:36.828209 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a8b4f3-a758-4834-afaf-92fdd7525a83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:36.829234 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a8b4f3-a758-4834-afaf-92fdd7525a83 None admin] RBAC: Adding query filter params (role.id=1d3b684bd8714b5792ebd7695c02a9ab, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=c48d41dcf9b344a8b2fec77f4dd75fcb) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:36.829551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a8b4f3-a758-4834-afaf-92fdd7525a83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:36.831005 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a8b4f3-a758-4834-afaf-92fdd7525a83 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:36.837320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 549/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:57:36 2022] GET /identity/v3/role_assignments?user.id=c48d41dcf9b344a8b2fec77f4dd75fcb&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=1d3b684bd8714b5792ebd7695c02a9ab => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:38.017616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb1342-f51d-4aeb-ac50-73862ac88a8c None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:38.018142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb1342-f51d-4aeb-ac50-73862ac88a8c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:38.018142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb1342-f51d-4aeb-ac50-73862ac88a8c None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:38.019428 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 558/1107] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:38.024727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57660358-95a1-43e2-b11d-4f560c8065e5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:38.025133 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57660358-95a1-43e2-b11d-4f560c8065e5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:38.025286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57660358-95a1-43e2-b11d-4f560c8065e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:38.041819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-57660358-95a1-43e2-b11d-4f560c8065e5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:38.043003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-57660358-95a1-43e2-b11d-4f560c8065e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:38.109425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 550/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:38.135593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb1342-f51d-4aeb-ac50-73862ac88a8c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:38.160187 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:38.244620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb1342-f51d-4aeb-ac50-73862ac88a8c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:38.278943 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44407cbc-5d28-4fa6-8e96-cd8d3bedda7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:38.279652 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44407cbc-5d28-4fa6-8e96-cd8d3bedda7c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:38.279956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44407cbc-5d28-4fa6-8e96-cd8d3bedda7c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:38.280237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44407cbc-5d28-4fa6-8e96-cd8d3bedda7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:38.281403 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44407cbc-5d28-4fa6-8e96-cd8d3bedda7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:38.283369 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44407cbc-5d28-4fa6-8e96-cd8d3bedda7c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:38.284999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 559/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:38.290867 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57660358-95a1-43e2-b11d-4f560c8065e5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:38.306638 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:38.392152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57660358-95a1-43e2-b11d-4f560c8065e5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:38.430647 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba299e27-7548-4876-b09f-b8021d75475a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:38.431330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba299e27-7548-4876-b09f-b8021d75475a None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:38.431617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba299e27-7548-4876-b09f-b8021d75475a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:38.431895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba299e27-7548-4876-b09f-b8021d75475a None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:38.432660 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba299e27-7548-4876-b09f-b8021d75475a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:38.434132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba299e27-7548-4876-b09f-b8021d75475a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:38.454947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 551/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:57:38 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:57:40.538628 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c64c7-863d-4939-b2d4-73632b58e08c None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:40.539228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c64c7-863d-4939-b2d4-73632b58e08c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:40.539228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c64c7-863d-4939-b2d4-73632b58e08c None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:40.540554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 560/1111] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:40.545927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1c8b7-5f80-4d67-95a2-930a4da8d4f0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:40.546356 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1c8b7-5f80-4d67-95a2-930a4da8d4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:40.546485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1c8b7-5f80-4d67-95a2-930a4da8d4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:40.563696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-98f1c8b7-5f80-4d67-95a2-930a4da8d4f0 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:40.564905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-98f1c8b7-5f80-4d67-95a2-930a4da8d4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:40.626448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:40.632040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 552/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:40.655756 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c64c7-863d-4939-b2d4-73632b58e08c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:40.747067 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:40.757760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c64c7-863d-4939-b2d4-73632b58e08c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:40.791317 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745c307-2fe1-47c9-8916-3c04d2ed02fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:40.792120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745c307-2fe1-47c9-8916-3c04d2ed02fb None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:40.792407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745c307-2fe1-47c9-8916-3c04d2ed02fb None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:40.792685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745c307-2fe1-47c9-8916-3c04d2ed02fb None admin] PATH_INFO: `/v3/roles/5a989bb8c8a1465ca76622ffb504dee5` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:40.794161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745c307-2fe1-47c9-8916-3c04d2ed02fb None admin] RBAC: Authorizing `identity:get_role(role_id=5a989bb8c8a1465ca76622ffb504dee5)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:40.795534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745c307-2fe1-47c9-8916-3c04d2ed02fb None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:40.796928 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 561/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:57:40 2022] GET /identity/v3/roles/5a989bb8c8a1465ca76622ffb504dee5 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:40.803049 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1c8b7-5f80-4d67-95a2-930a4da8d4f0 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:40.900221 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1c8b7-5f80-4d67-95a2-930a4da8d4f0 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:40.933910 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db861a-05b3-4f11-a3a5-bb6ef0fb08ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:40.934798 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db861a-05b3-4f11-a3a5-bb6ef0fb08ce None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:40.935089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db861a-05b3-4f11-a3a5-bb6ef0fb08ce None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:40.935367 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db861a-05b3-4f11-a3a5-bb6ef0fb08ce None admin] PATH_INFO: `/v3/users/573ae32d23274a2ba5acc5ab3739e93b` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:40.948833 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db861a-05b3-4f11-a3a5-bb6ef0fb08ce None admin] RBAC: Authorizing `identity:get_user(user_id=573ae32d23274a2ba5acc5ab3739e93b)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:40.950293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db861a-05b3-4f11-a3a5-bb6ef0fb08ce None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:40.951957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 553/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:57:40 2022] GET /identity/v3/users/573ae32d23274a2ba5acc5ab3739e93b => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:40.958313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745c307-2fe1-47c9-8916-3c04d2ed02fb None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:41.055797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745c307-2fe1-47c9-8916-3c04d2ed02fb None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:41.089387 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a70947-e8ad-4504-888a-89389fe93d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:41.090022 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a70947-e8ad-4504-888a-89389fe93d8e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:41.090310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a70947-e8ad-4504-888a-89389fe93d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:41.090589 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a70947-e8ad-4504-888a-89389fe93d8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:41.117249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:41.158285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a70947-e8ad-4504-888a-89389fe93d8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:41.159734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a70947-e8ad-4504-888a-89389fe93d8e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:41.258524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 562/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:57:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:57:41.264428 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db861a-05b3-4f11-a3a5-bb6ef0fb08ce None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:41.321765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:41.362356 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db861a-05b3-4f11-a3a5-bb6ef0fb08ce None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:41.394249 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef6d25-ff3d-486b-b5f3-f7dca3939d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:41.395084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef6d25-ff3d-486b-b5f3-f7dca3939d23 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:41.395362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef6d25-ff3d-486b-b5f3-f7dca3939d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:41.395633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef6d25-ff3d-486b-b5f3-f7dca3939d23 None admin] PATH_INFO: `/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:41.396808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef6d25-ff3d-486b-b5f3-f7dca3939d23 None admin] RBAC: Authorizing `identity:get_project(project_id=4dc8f87e239a4cbfacfcfc057f00bd7d)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:41.398229 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef6d25-ff3d-486b-b5f3-f7dca3939d23 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:41.399887 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 554/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:57:41 2022] GET /identity/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:41.405942 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a70947-e8ad-4504-888a-89389fe93d8e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:41.462766 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:41.505641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a70947-e8ad-4504-888a-89389fe93d8e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:41.539816 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d13855c-ee39-4221-a2b7-3e5814916ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:41.540493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13855c-ee39-4221-a2b7-3e5814916ada None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:41.540778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13855c-ee39-4221-a2b7-3e5814916ada None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:41.541053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13855c-ee39-4221-a2b7-3e5814916ada None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:41.541875 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13855c-ee39-4221-a2b7-3e5814916ada None admin] RBAC: Adding query filter params (role.id=5a989bb8c8a1465ca76622ffb504dee5, scope.project.id=4dc8f87e239a4cbfacfcfc057f00bd7d, user.id=573ae32d23274a2ba5acc5ab3739e93b) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:41.542182 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13855c-ee39-4221-a2b7-3e5814916ada None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:41.543608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d13855c-ee39-4221-a2b7-3e5814916ada None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:41.549353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 563/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:57:41 2022] GET /identity/v3/role_assignments?user.id=573ae32d23274a2ba5acc5ab3739e93b&scope.project.id=4dc8f87e239a4cbfacfcfc057f00bd7d&role.id=5a989bb8c8a1465ca76622ffb504dee5 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:43.661841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12fd1b-7746-4bc6-bbba-51ff62fdd3c6 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:43.662431 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12fd1b-7746-4bc6-bbba-51ff62fdd3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:43.662431 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12fd1b-7746-4bc6-bbba-51ff62fdd3c6 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:43.663829 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 555/1118] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:43.670115 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de214f-2ffb-46bd-b4e5-61f4fa624cc7 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:43.670715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de214f-2ffb-46bd-b4e5-61f4fa624cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:43.670715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de214f-2ffb-46bd-b4e5-61f4fa624cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:43.690759 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e4de214f-2ffb-46bd-b4e5-61f4fa624cc7 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:43.692023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e4de214f-2ffb-46bd-b4e5-61f4fa624cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:43.768640 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 564/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:43.792547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12fd1b-7746-4bc6-bbba-51ff62fdd3c6 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:43.904253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12fd1b-7746-4bc6-bbba-51ff62fdd3c6 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:43.934715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:43.946033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9897b31-8b7f-40ee-ae0c-7eae31894074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:43.946893 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9897b31-8b7f-40ee-ae0c-7eae31894074 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:43.947177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9897b31-8b7f-40ee-ae0c-7eae31894074 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:43.947448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9897b31-8b7f-40ee-ae0c-7eae31894074 None admin] PATH_INFO: `/v3/roles/5a989bb8c8a1465ca76622ffb504dee5` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:43.948990 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9897b31-8b7f-40ee-ae0c-7eae31894074 None admin] RBAC: Authorizing `identity:get_role(role_id=5a989bb8c8a1465ca76622ffb504dee5)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:43.950377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9897b31-8b7f-40ee-ae0c-7eae31894074 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:43.951776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 556/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:57:43 2022] GET /identity/v3/roles/5a989bb8c8a1465ca76622ffb504dee5 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:43.960519 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de214f-2ffb-46bd-b4e5-61f4fa624cc7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:44.022511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:44.083159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de214f-2ffb-46bd-b4e5-61f4fa624cc7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:44.121628 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4623e-e523-4831-a03d-dc7c84dee44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:44.122495 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4623e-e523-4831-a03d-dc7c84dee44c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:44.122778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4623e-e523-4831-a03d-dc7c84dee44c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:44.123052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4623e-e523-4831-a03d-dc7c84dee44c None admin] PATH_INFO: `/v3/users/573ae32d23274a2ba5acc5ab3739e93b` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:44.124292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4623e-e523-4831-a03d-dc7c84dee44c None admin] RBAC: Authorizing `identity:get_user(user_id=573ae32d23274a2ba5acc5ab3739e93b)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:44.127036 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4623e-e523-4831-a03d-dc7c84dee44c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:44.128760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 565/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:57:43 2022] GET /identity/v3/users/573ae32d23274a2ba5acc5ab3739e93b => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:44.134980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9897b31-8b7f-40ee-ae0c-7eae31894074 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:44.248925 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9897b31-8b7f-40ee-ae0c-7eae31894074 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:44.288375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bdcca-5b78-47d9-a3a8-900b10a5b06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:44.289253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bdcca-5b78-47d9-a3a8-900b10a5b06b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:44.289540 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bdcca-5b78-47d9-a3a8-900b10a5b06b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:44.289818 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bdcca-5b78-47d9-a3a8-900b10a5b06b None admin] PATH_INFO: `/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:44.291040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bdcca-5b78-47d9-a3a8-900b10a5b06b None admin] RBAC: Authorizing `identity:get_project(project_id=4dc8f87e239a4cbfacfcfc057f00bd7d)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:44.292551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bdcca-5b78-47d9-a3a8-900b10a5b06b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:44.294315 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 557/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:57:44 2022] GET /identity/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:44.302263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4623e-e523-4831-a03d-dc7c84dee44c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:44.418901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4623e-e523-4831-a03d-dc7c84dee44c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:44.441974 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:44.457446 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa19ef-9ac3-40ea-a379-b8b1e18a75c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:44.458342 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa19ef-9ac3-40ea-a379-b8b1e18a75c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:44.458629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa19ef-9ac3-40ea-a379-b8b1e18a75c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:44.458907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa19ef-9ac3-40ea-a379-b8b1e18a75c5 None admin] PATH_INFO: `/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d/users/573ae32d23274a2ba5acc5ab3739e93b/roles/5a989bb8c8a1465ca76622ffb504dee5` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:44.461236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa19ef-9ac3-40ea-a379-b8b1e18a75c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4dc8f87e239a4cbfacfcfc057f00bd7d, user_id=573ae32d23274a2ba5acc5ab3739e93b, role_id=5a989bb8c8a1465ca76622ffb504dee5)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:44.462707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa19ef-9ac3-40ea-a379-b8b1e18a75c5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:44.476905 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 566/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:57:44 2022] PUT /identity/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d/users/573ae32d23274a2ba5acc5ab3739e93b/roles/5a989bb8c8a1465ca76622ffb504dee5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:57:46.689607 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3b395-92e8-42c7-a2ab-f78f4229abba None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:46.690177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3b395-92e8-42c7-a2ab-f78f4229abba None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:46.690177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3b395-92e8-42c7-a2ab-f78f4229abba None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:46.691549 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 558/1124] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:46.701533 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fbbfa-8ffa-4126-bbbb-acfe8eb628bd None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:46.702059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fbbfa-8ffa-4126-bbbb-acfe8eb628bd None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:46.702517 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fbbfa-8ffa-4126-bbbb-acfe8eb628bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:46.730586 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-037fbbfa-8ffa-4126-bbbb-acfe8eb628bd None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:46.732370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-037fbbfa-8ffa-4126-bbbb-acfe8eb628bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:46.841636 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 567/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:46.867732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3b395-92e8-42c7-a2ab-f78f4229abba None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:46.968378 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:47.020303 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3b395-92e8-42c7-a2ab-f78f4229abba None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:47.058462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686086b-1585-451a-a1e0-7f866c9fa0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:47.059315 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686086b-1585-451a-a1e0-7f866c9fa0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:47.059597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686086b-1585-451a-a1e0-7f866c9fa0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:47.060020 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686086b-1585-451a-a1e0-7f866c9fa0f4 None admin] PATH_INFO: `/v3/roles/5a989bb8c8a1465ca76622ffb504dee5` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:47.061503 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686086b-1585-451a-a1e0-7f866c9fa0f4 None admin] RBAC: Authorizing `identity:get_role(role_id=5a989bb8c8a1465ca76622ffb504dee5)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:47.063503 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686086b-1585-451a-a1e0-7f866c9fa0f4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:47.065148 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 559/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:57:46 2022] GET /identity/v3/roles/5a989bb8c8a1465ca76622ffb504dee5 => generated 223 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:47.072794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037fbbfa-8ffa-4126-bbbb-acfe8eb628bd None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:47.188173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037fbbfa-8ffa-4126-bbbb-acfe8eb628bd None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:47.200368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:47.229801 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126a0d2-1797-4b84-a543-85b728099dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:47.230609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126a0d2-1797-4b84-a543-85b728099dcd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:47.230916 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126a0d2-1797-4b84-a543-85b728099dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:47.231212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126a0d2-1797-4b84-a543-85b728099dcd None admin] PATH_INFO: `/v3/users/573ae32d23274a2ba5acc5ab3739e93b` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:47.232536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126a0d2-1797-4b84-a543-85b728099dcd None admin] RBAC: Authorizing `identity:get_user(user_id=573ae32d23274a2ba5acc5ab3739e93b)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:47.234106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126a0d2-1797-4b84-a543-85b728099dcd None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:47.235888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 568/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:57:47 2022] GET /identity/v3/users/573ae32d23274a2ba5acc5ab3739e93b => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:47.242092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686086b-1585-451a-a1e0-7f866c9fa0f4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:47.399233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686086b-1585-451a-a1e0-7f866c9fa0f4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:47.449164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:47.459059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edd208-e85b-4242-945b-3a38c2fa8e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:47.460190 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edd208-e85b-4242-945b-3a38c2fa8e9e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:47.460674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edd208-e85b-4242-945b-3a38c2fa8e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:47.461141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edd208-e85b-4242-945b-3a38c2fa8e9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:47.570613 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edd208-e85b-4242-945b-3a38c2fa8e9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:47.573091 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edd208-e85b-4242-945b-3a38c2fa8e9e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:47.691426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 560/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:57:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:57:47.697705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126a0d2-1797-4b84-a543-85b728099dcd None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:47.811461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126a0d2-1797-4b84-a543-85b728099dcd None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:47.852808 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa10f7-a0de-49b9-87d3-3fe1382addef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:47.853678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa10f7-a0de-49b9-87d3-3fe1382addef None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:47.854030 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa10f7-a0de-49b9-87d3-3fe1382addef None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:47.854352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa10f7-a0de-49b9-87d3-3fe1382addef None admin] PATH_INFO: `/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:47.855674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa10f7-a0de-49b9-87d3-3fe1382addef None admin] RBAC: Authorizing `identity:get_project(project_id=4dc8f87e239a4cbfacfcfc057f00bd7d)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:47.857405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa10f7-a0de-49b9-87d3-3fe1382addef None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:47.859312 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 569/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:57:47 2022] GET /identity/v3/projects/4dc8f87e239a4cbfacfcfc057f00bd7d => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:47.866473 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edd208-e85b-4242-945b-3a38c2fa8e9e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:47.901399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:48.001151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edd208-e85b-4242-945b-3a38c2fa8e9e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:48.039995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74434478-b8c0-4c97-8a6a-0ac9da0eab13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:48.040658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74434478-b8c0-4c97-8a6a-0ac9da0eab13 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:48.040940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74434478-b8c0-4c97-8a6a-0ac9da0eab13 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:48.041214 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74434478-b8c0-4c97-8a6a-0ac9da0eab13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:48.042059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74434478-b8c0-4c97-8a6a-0ac9da0eab13 None admin] RBAC: Adding query filter params (role.id=5a989bb8c8a1465ca76622ffb504dee5, scope.project.id=4dc8f87e239a4cbfacfcfc057f00bd7d, user.id=573ae32d23274a2ba5acc5ab3739e93b) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:48.042375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74434478-b8c0-4c97-8a6a-0ac9da0eab13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:48.043858 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74434478-b8c0-4c97-8a6a-0ac9da0eab13 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:48.052488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 561/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:57:47 2022] GET /identity/v3/role_assignments?user.id=573ae32d23274a2ba5acc5ab3739e93b&scope.project.id=4dc8f87e239a4cbfacfcfc057f00bd7d&role.id=5a989bb8c8a1465ca76622ffb504dee5 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:50.343664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706908bb-b30c-45d2-bb92-145e2e66d392 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:50.344272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706908bb-b30c-45d2-bb92-145e2e66d392 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:50.344272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706908bb-b30c-45d2-bb92-145e2e66d392 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:50.345748 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 570/1131] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:50.352319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1e1f7-c23f-4e80-a0df-28099781e8a9 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:50.352865 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1e1f7-c23f-4e80-a0df-28099781e8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:50.352954 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1e1f7-c23f-4e80-a0df-28099781e8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:50.371052 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e0f1e1f7-c23f-4e80-a0df-28099781e8a9 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:50.372322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e0f1e1f7-c23f-4e80-a0df-28099781e8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:50.453521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 562/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:50.483032 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706908bb-b30c-45d2-bb92-145e2e66d392 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:50.542619 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:50.612382 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706908bb-b30c-45d2-bb92-145e2e66d392 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:50.658647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c87c3-31e0-416f-8fb6-9dcbf76b683e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:50.660140 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c87c3-31e0-416f-8fb6-9dcbf76b683e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:50.660645 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c87c3-31e0-416f-8fb6-9dcbf76b683e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:50.660992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c87c3-31e0-416f-8fb6-9dcbf76b683e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:50.662655 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c87c3-31e0-416f-8fb6-9dcbf76b683e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:50.664199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c87c3-31e0-416f-8fb6-9dcbf76b683e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:50.665986 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 571/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:50.672325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1e1f7-c23f-4e80-a0df-28099781e8a9 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:50.705771 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:50.802153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1e1f7-c23f-4e80-a0df-28099781e8a9 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:50.857843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6ce0c-64b7-4220-99af-b0c5c4c60857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:50.858828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6ce0c-64b7-4220-99af-b0c5c4c60857 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:50.859184 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6ce0c-64b7-4220-99af-b0c5c4c60857 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:50.859530 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6ce0c-64b7-4220-99af-b0c5c4c60857 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:50.860927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6ce0c-64b7-4220-99af-b0c5c4c60857 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:50.862620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b6ce0c-64b7-4220-99af-b0c5c4c60857 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:50.899678 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 563/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jul 21 08:57:50 2022] POST /identity/v3/projects => generated 311 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:57:53.101516 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ce194-71f2-4f20-86d1-ce2ea95e35aa None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:53.102101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ce194-71f2-4f20-86d1-ce2ea95e35aa None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:53.102101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ce194-71f2-4f20-86d1-ce2ea95e35aa None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:53.103555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 572/1135] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:53.109446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c3886-6f22-4f36-986b-ceff571326ff None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:53.110055 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c3886-6f22-4f36-986b-ceff571326ff None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:53.110971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c3886-6f22-4f36-986b-ceff571326ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:53.129592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-eb1c3886-6f22-4f36-986b-ceff571326ff None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:53.130935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-eb1c3886-6f22-4f36-986b-ceff571326ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:53.208035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 564/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:53.234065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ce194-71f2-4f20-86d1-ce2ea95e35aa None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:53.348515 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ce194-71f2-4f20-86d1-ce2ea95e35aa None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:53.364047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:53.386291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f923fd-3551-41c8-b313-0031d615a270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:53.387092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f923fd-3551-41c8-b313-0031d615a270 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:53.387376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f923fd-3551-41c8-b313-0031d615a270 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:53.387649 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f923fd-3551-41c8-b313-0031d615a270 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:53.388911 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f923fd-3551-41c8-b313-0031d615a270 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:53.390377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f923fd-3551-41c8-b313-0031d615a270 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:53.392676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 573/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:57:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:53.402434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c3886-6f22-4f36-986b-ceff571326ff None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:53.412079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:53.514256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c3886-6f22-4f36-986b-ceff571326ff None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:53.553314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf48b8c-c68a-4766-9445-91f5c0c25fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:53.553988 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf48b8c-c68a-4766-9445-91f5c0c25fda None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:53.554274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf48b8c-c68a-4766-9445-91f5c0c25fda None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:53.554552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf48b8c-c68a-4766-9445-91f5c0c25fda None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:53.555284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf48b8c-c68a-4766-9445-91f5c0c25fda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:53.556782 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf48b8c-c68a-4766-9445-91f5c0c25fda None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:53.580422 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 565/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:57:53 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:57:55.846476 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a65c0e-ad61-4710-ad70-7e04261f8bbb None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:55.847015 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a65c0e-ad61-4710-ad70-7e04261f8bbb None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:55.847454 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a65c0e-ad61-4710-ad70-7e04261f8bbb None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:55.849979 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 574/1139] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:55.857452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9fa83-0fd6-44c4-8597-e5b05e0d91b5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:55.858044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9fa83-0fd6-44c4-8597-e5b05e0d91b5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:55.858044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9fa83-0fd6-44c4-8597-e5b05e0d91b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:55.878019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-44c9fa83-0fd6-44c4-8597-e5b05e0d91b5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:55.879294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-44c9fa83-0fd6-44c4-8597-e5b05e0d91b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:55.943397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:55.955796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 566/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:57:55.980089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a65c0e-ad61-4710-ad70-7e04261f8bbb None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:56.099613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a65c0e-ad61-4710-ad70-7e04261f8bbb None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:56.136629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c61d8e-64e7-4bdf-b5bc-b61bea7dce14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:56.137519 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c61d8e-64e7-4bdf-b5bc-b61bea7dce14 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:56.137809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c61d8e-64e7-4bdf-b5bc-b61bea7dce14 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:56.138171 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c61d8e-64e7-4bdf-b5bc-b61bea7dce14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:56.151239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c61d8e-64e7-4bdf-b5bc-b61bea7dce14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-a7c61d8e-64e7-4bdf-b5bc-b61bea7dce14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:57:56.160715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:56.161914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:56.162890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:56.162890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:56.162890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:56.162890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:56.162890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:57:56.162890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:57:56.162890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:57:56.162890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:57:56.162890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:57:56.162890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:57:56.162890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:57:56.162890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:56.162890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 575/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:57:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:56.168716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9fa83-0fd6-44c4-8597-e5b05e0d91b5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:56.292965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9fa83-0fd6-44c4-8597-e5b05e0d91b5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:56.332487 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84043e97-8e7e-42c2-b0b2-6ee06b2fa81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:56.333162 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84043e97-8e7e-42c2-b0b2-6ee06b2fa81f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:56.333493 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84043e97-8e7e-42c2-b0b2-6ee06b2fa81f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:56.333790 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84043e97-8e7e-42c2-b0b2-6ee06b2fa81f None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:56.334539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84043e97-8e7e-42c2-b0b2-6ee06b2fa81f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:56.334844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84043e97-8e7e-42c2-b0b2-6ee06b2fa81f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:56.336255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84043e97-8e7e-42c2-b0b2-6ee06b2fa81f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:56.345534 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 567/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:57:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:56.353429 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c61d8e-64e7-4bdf-b5bc-b61bea7dce14 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:56.409963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:56.518478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c61d8e-64e7-4bdf-b5bc-b61bea7dce14 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:56.579465 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01a0e9-9a5b-4061-a8af-68e389b93fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:56.581042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01a0e9-9a5b-4061-a8af-68e389b93fad None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:56.581558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01a0e9-9a5b-4061-a8af-68e389b93fad None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:56.582070 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01a0e9-9a5b-4061-a8af-68e389b93fad None admin] PATH_INFO: `/v3/users/1b2f518914314e95ada0ba8b2a9fadb8` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:56.608836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01a0e9-9a5b-4061-a8af-68e389b93fad None admin] RBAC: Authorizing `identity:get_user(user_id=1b2f518914314e95ada0ba8b2a9fadb8)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:56.612418 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01a0e9-9a5b-4061-a8af-68e389b93fad None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:56.615736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 576/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:57:56 2022] GET /identity/v3/users/1b2f518914314e95ada0ba8b2a9fadb8 => generated 286 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:56.622423 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84043e97-8e7e-42c2-b0b2-6ee06b2fa81f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:56.690550 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:56.747757 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84043e97-8e7e-42c2-b0b2-6ee06b2fa81f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:56.783078 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e287f79-4d6d-4ce1-99cb-66c9fdeb4009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:56.783715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e287f79-4d6d-4ce1-99cb-66c9fdeb4009 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:56.784022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e287f79-4d6d-4ce1-99cb-66c9fdeb4009 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:56.784297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e287f79-4d6d-4ce1-99cb-66c9fdeb4009 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:56.855238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e287f79-4d6d-4ce1-99cb-66c9fdeb4009 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:56.856736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e287f79-4d6d-4ce1-99cb-66c9fdeb4009 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:56.914088 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:56.971660 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 568/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:57:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:57:56.980991 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01a0e9-9a5b-4061-a8af-68e389b93fad None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:57.099369 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:57.102620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01a0e9-9a5b-4061-a8af-68e389b93fad None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:57.139476 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904c7f7-617b-450c-bbf1-f721e9a7e395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:57.140356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904c7f7-617b-450c-bbf1-f721e9a7e395 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:57.140634 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904c7f7-617b-450c-bbf1-f721e9a7e395 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:57.140902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904c7f7-617b-450c-bbf1-f721e9a7e395 None admin] PATH_INFO: `/v3/projects/994704cf8c54438db6ca9067c82f747d` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:57.142144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904c7f7-617b-450c-bbf1-f721e9a7e395 None admin] RBAC: Authorizing `identity:get_project(project_id=994704cf8c54438db6ca9067c82f747d)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:57.143600 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904c7f7-617b-450c-bbf1-f721e9a7e395 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:57.145328 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 577/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:57:56 2022] GET /identity/v3/projects/994704cf8c54438db6ca9067c82f747d => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:57.153341 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e287f79-4d6d-4ce1-99cb-66c9fdeb4009 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:57.258926 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e287f79-4d6d-4ce1-99cb-66c9fdeb4009 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:57.294958 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308678b-486a-418a-bfc5-4fd302d4f894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:57.295620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308678b-486a-418a-bfc5-4fd302d4f894 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:57.295937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308678b-486a-418a-bfc5-4fd302d4f894 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:57.296217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308678b-486a-418a-bfc5-4fd302d4f894 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:57.297052 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308678b-486a-418a-bfc5-4fd302d4f894 None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=994704cf8c54438db6ca9067c82f747d, user.id=1b2f518914314e95ada0ba8b2a9fadb8) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:57.297401 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308678b-486a-418a-bfc5-4fd302d4f894 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:57.299711 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308678b-486a-418a-bfc5-4fd302d4f894 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:57.306657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 569/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:57:57 2022] GET /identity/v3/role_assignments?user.id=1b2f518914314e95ada0ba8b2a9fadb8&scope.project.id=994704cf8c54438db6ca9067c82f747d&role.id=000e942517f2469ba36d2e65978bc314 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:57:59.472394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d948248-68eb-43b0-b298-7c87d2fe15e3 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:59.472934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d948248-68eb-43b0-b298-7c87d2fe15e3 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:59.472934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d948248-68eb-43b0-b298-7c87d2fe15e3 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:59.474297 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 578/1147] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:57:59.480287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449a597-326a-4e8a-bdbf-bc53bbcbaf86 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:59.480736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449a597-326a-4e8a-bdbf-bc53bbcbaf86 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:59.480857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449a597-326a-4e8a-bdbf-bc53bbcbaf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:59.499351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4449a597-326a-4e8a-bdbf-bc53bbcbaf86 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:57:59.501348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4449a597-326a-4e8a-bdbf-bc53bbcbaf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:57:59.517525 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:57:59.578893 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 570/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:57:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:57:59.602953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d948248-68eb-43b0-b298-7c87d2fe15e3 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:59.719069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d948248-68eb-43b0-b298-7c87d2fe15e3 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:59.760269 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd4d93-0cf4-41ee-828d-5971f42ccb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:59.760980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd4d93-0cf4-41ee-828d-5971f42ccb8c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:59.761266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd4d93-0cf4-41ee-828d-5971f42ccb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:59.761627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd4d93-0cf4-41ee-828d-5971f42ccb8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:59.774735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd4d93-0cf4-41ee-828d-5971f42ccb8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-40dd4d93-0cf4-41ee-828d-5971f42ccb8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:57:59.783653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:57:59.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:57:59.786027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:57:59.786027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:57:59.786027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:57:59.786027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:57:59.786027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:57:59.786027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:57:59.786027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:57:59.786027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:57:59.786027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:57:59.786027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:57:59.786027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:57:59.786027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:57:59.786027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 579/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:57:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:57:59.791334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449a597-326a-4e8a-bdbf-bc53bbcbaf86 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:57:59.911192 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449a597-326a-4e8a-bdbf-bc53bbcbaf86 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:57:59.949114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784cd23b-314b-4761-b3a8-3649e09cab68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:57:59.949843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cd23b-314b-4761-b3a8-3649e09cab68 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:57:59.950124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cd23b-314b-4761-b3a8-3649e09cab68 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:57:59.950432 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cd23b-314b-4761-b3a8-3649e09cab68 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:57:59.951213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784cd23b-314b-4761-b3a8-3649e09cab68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:57:59.951541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784cd23b-314b-4761-b3a8-3649e09cab68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:57:59.952972 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784cd23b-314b-4761-b3a8-3649e09cab68 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:57:59.962193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 571/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:57:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:57:59.968685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd4d93-0cf4-41ee-828d-5971f42ccb8c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:00.002630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:00.090429 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd4d93-0cf4-41ee-828d-5971f42ccb8c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:00.128258 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18705d92-d0f1-42f0-a1ba-cc20fa49e5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:00.129737 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18705d92-d0f1-42f0-a1ba-cc20fa49e5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:00.130236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18705d92-d0f1-42f0-a1ba-cc20fa49e5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:00.130731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18705d92-d0f1-42f0-a1ba-cc20fa49e5b6 None admin] PATH_INFO: `/v3/users/1b2f518914314e95ada0ba8b2a9fadb8` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:00.132794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18705d92-d0f1-42f0-a1ba-cc20fa49e5b6 None admin] RBAC: Authorizing `identity:get_user(user_id=1b2f518914314e95ada0ba8b2a9fadb8)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:00.135455 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18705d92-d0f1-42f0-a1ba-cc20fa49e5b6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:00.138680 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 580/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:57:59 2022] GET /identity/v3/users/1b2f518914314e95ada0ba8b2a9fadb8 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:00.145174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784cd23b-314b-4761-b3a8-3649e09cab68 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:00.158881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:00.259536 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784cd23b-314b-4761-b3a8-3649e09cab68 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:00.300919 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a3aff-6bb7-4818-8b24-62f8a74b71e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:00.301782 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a3aff-6bb7-4818-8b24-62f8a74b71e6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:00.302071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a3aff-6bb7-4818-8b24-62f8a74b71e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:00.302353 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a3aff-6bb7-4818-8b24-62f8a74b71e6 None admin] PATH_INFO: `/v3/projects/994704cf8c54438db6ca9067c82f747d` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:00.303577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a3aff-6bb7-4818-8b24-62f8a74b71e6 None admin] RBAC: Authorizing `identity:get_project(project_id=994704cf8c54438db6ca9067c82f747d)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:00.305072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a3aff-6bb7-4818-8b24-62f8a74b71e6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:00.306803 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 572/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:58:00 2022] GET /identity/v3/projects/994704cf8c54438db6ca9067c82f747d => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:58:00.313451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18705d92-d0f1-42f0-a1ba-cc20fa49e5b6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:00.420584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:00.429213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18705d92-d0f1-42f0-a1ba-cc20fa49e5b6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:00.465552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b3da9-dcd1-43c5-b354-2c94fd712a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:00.466441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b3da9-dcd1-43c5-b354-2c94fd712a64 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:00.466724 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b3da9-dcd1-43c5-b354-2c94fd712a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:00.467003 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b3da9-dcd1-43c5-b354-2c94fd712a64 None admin] PATH_INFO: `/v3/projects/994704cf8c54438db6ca9067c82f747d/users/1b2f518914314e95ada0ba8b2a9fadb8/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:00.469121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b3da9-dcd1-43c5-b354-2c94fd712a64 None admin] RBAC: Authorizing `identity:create_grant(project_id=994704cf8c54438db6ca9067c82f747d, user_id=1b2f518914314e95ada0ba8b2a9fadb8, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:00.471027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b3da9-dcd1-43c5-b354-2c94fd712a64 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:00.485953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 581/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:58:00 2022] PUT /identity/v3/projects/994704cf8c54438db6ca9067c82f747d/users/1b2f518914314e95ada0ba8b2a9fadb8/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:58:02.717868 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352d974-4d3d-41de-bb43-257e9164caa7 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:02.718447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352d974-4d3d-41de-bb43-257e9164caa7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:02.718447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352d974-4d3d-41de-bb43-257e9164caa7 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:02.721087 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 573/1154] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:58:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:58:02.727637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ea27c-6b82-4662-b842-c5a8cea5def2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:02.728184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ea27c-6b82-4662-b842-c5a8cea5def2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:02.728328 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ea27c-6b82-4662-b842-c5a8cea5def2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:02.756070 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-9e8ea27c-6b82-4662-b842-c5a8cea5def2 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:58:02.758007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9e8ea27c-6b82-4662-b842-c5a8cea5def2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:58:02.843127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 582/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:58:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:58:02.867302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352d974-4d3d-41de-bb43-257e9164caa7 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:02.951655 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:02.975584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352d974-4d3d-41de-bb43-257e9164caa7 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:03.012696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d171acf-f766-4845-bfa1-80625ec22672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:03.013491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d171acf-f766-4845-bfa1-80625ec22672 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:03.013776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d171acf-f766-4845-bfa1-80625ec22672 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:03.014052 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d171acf-f766-4845-bfa1-80625ec22672 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:03.026118 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d171acf-f766-4845-bfa1-80625ec22672 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-6d171acf-f766-4845-bfa1-80625ec22672 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:58:03.034414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:58:03.035551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:58:03.036500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:58:03.036500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:58:03.036500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:58:03.036500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:58:03.036500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:58:03.036500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:58:03.036500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:58:03.036500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:58:03.036500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:58:03.036500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:58:03.036500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:58:03.036500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:58:03.036500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 574/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:58:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:58:03.045983 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ea27c-6b82-4662-b842-c5a8cea5def2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:03.140878 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:03.159047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ea27c-6b82-4662-b842-c5a8cea5def2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:03.197700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edfa46-1f53-4c32-9355-78a0fbede75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:03.198394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edfa46-1f53-4c32-9355-78a0fbede75a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:03.198694 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edfa46-1f53-4c32-9355-78a0fbede75a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:03.198984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edfa46-1f53-4c32-9355-78a0fbede75a None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:03.199809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73edfa46-1f53-4c32-9355-78a0fbede75a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:58:03.200154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73edfa46-1f53-4c32-9355-78a0fbede75a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:03.201725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73edfa46-1f53-4c32-9355-78a0fbede75a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:03.211776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 583/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:58:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:03.218299 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d171acf-f766-4845-bfa1-80625ec22672 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:03.331241 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d171acf-f766-4845-bfa1-80625ec22672 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:03.369077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b533a8-2613-442b-9834-224562c0d9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:03.370037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b533a8-2613-442b-9834-224562c0d9df None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:03.370350 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b533a8-2613-442b-9834-224562c0d9df None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:03.370651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b533a8-2613-442b-9834-224562c0d9df None admin] PATH_INFO: `/v3/users/1b2f518914314e95ada0ba8b2a9fadb8` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:03.371995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b533a8-2613-442b-9834-224562c0d9df None admin] RBAC: Authorizing `identity:get_user(user_id=1b2f518914314e95ada0ba8b2a9fadb8)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:03.373505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b533a8-2613-442b-9834-224562c0d9df None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:03.375281 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 575/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:58:03 2022] GET /identity/v3/users/1b2f518914314e95ada0ba8b2a9fadb8 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:03.383320 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edfa46-1f53-4c32-9355-78a0fbede75a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:03.503698 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edfa46-1f53-4c32-9355-78a0fbede75a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:03.540770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0252f-9e49-4c0a-8205-b47a4ce4ecf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:03.541455 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0252f-9e49-4c0a-8205-b47a4ce4ecf1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:03.541737 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0252f-9e49-4c0a-8205-b47a4ce4ecf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:03.542091 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0252f-9e49-4c0a-8205-b47a4ce4ecf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:03.562639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:03.629638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0252f-9e49-4c0a-8205-b47a4ce4ecf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:03.631031 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0252f-9e49-4c0a-8205-b47a4ce4ecf1 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:03.740245 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 584/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:58:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:58:03.746262 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b533a8-2613-442b-9834-224562c0d9df None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:03.756960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:03.879861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b533a8-2613-442b-9834-224562c0d9df None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:03.916237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507e63f-8e4e-40da-833b-56e6d069e159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:03.917165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507e63f-8e4e-40da-833b-56e6d069e159 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:03.917387 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507e63f-8e4e-40da-833b-56e6d069e159 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:03.917752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507e63f-8e4e-40da-833b-56e6d069e159 None admin] PATH_INFO: `/v3/projects/994704cf8c54438db6ca9067c82f747d` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:03.918908 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507e63f-8e4e-40da-833b-56e6d069e159 None admin] RBAC: Authorizing `identity:get_project(project_id=994704cf8c54438db6ca9067c82f747d)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:03.920411 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507e63f-8e4e-40da-833b-56e6d069e159 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:03.922146 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 576/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:58:03 2022] GET /identity/v3/projects/994704cf8c54438db6ca9067c82f747d => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:03.929998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0252f-9e49-4c0a-8205-b47a4ce4ecf1 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:03.983566 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:04.038388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0252f-9e49-4c0a-8205-b47a4ce4ecf1 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:04.075664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752e82c-7262-4e71-b5bf-bc43e405603b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:04.076398 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752e82c-7262-4e71-b5bf-bc43e405603b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:04.076690 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752e82c-7262-4e71-b5bf-bc43e405603b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:04.076970 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752e82c-7262-4e71-b5bf-bc43e405603b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:04.077810 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752e82c-7262-4e71-b5bf-bc43e405603b None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=994704cf8c54438db6ca9067c82f747d, user.id=1b2f518914314e95ada0ba8b2a9fadb8) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:58:04.078124 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752e82c-7262-4e71-b5bf-bc43e405603b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:04.079788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752e82c-7262-4e71-b5bf-bc43e405603b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:04.086222 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 585/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:58:03 2022] GET /identity/v3/role_assignments?user.id=1b2f518914314e95ada0ba8b2a9fadb8&scope.project.id=994704cf8c54438db6ca9067c82f747d&role.id=000e942517f2469ba36d2e65978bc314 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:58:06.432887 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f77f09-a5ed-4185-b31d-735c145a98bc None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:06.433490 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f77f09-a5ed-4185-b31d-735c145a98bc None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:06.433616 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f77f09-a5ed-4185-b31d-735c145a98bc None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:06.435103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 577/1162] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:58:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:58:06.442340 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069099d5-2c30-4330-bd5b-052e59178505 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:06.442870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069099d5-2c30-4330-bd5b-052e59178505 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:06.442984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069099d5-2c30-4330-bd5b-052e59178505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:06.461970 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-069099d5-2c30-4330-bd5b-052e59178505 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:58:06.463178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-069099d5-2c30-4330-bd5b-052e59178505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:58:06.537850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 586/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:58:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:58:06.562074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f77f09-a5ed-4185-b31d-735c145a98bc None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:06.654051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:06.686447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f77f09-a5ed-4185-b31d-735c145a98bc None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:06.727767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acfc9e-f5f4-4c30-8905-30324b9e718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:06.728529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acfc9e-f5f4-4c30-8905-30324b9e718c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:06.728852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acfc9e-f5f4-4c30-8905-30324b9e718c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:06.729192 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acfc9e-f5f4-4c30-8905-30324b9e718c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:06.804373 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acfc9e-f5f4-4c30-8905-30324b9e718c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:06.805981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acfc9e-f5f4-4c30-8905-30324b9e718c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:06.888978 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:06.917774 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 578/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:58:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:58:06.925771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069099d5-2c30-4330-bd5b-052e59178505 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:06.997441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:07.066819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069099d5-2c30-4330-bd5b-052e59178505 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:07.105410 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d2103-1579-4603-84ef-1fd4d2e3424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:07.106222 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d2103-1579-4603-84ef-1fd4d2e3424d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:07.106508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d2103-1579-4603-84ef-1fd4d2e3424d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:07.106785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d2103-1579-4603-84ef-1fd4d2e3424d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:07.115816 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d2103-1579-4603-84ef-1fd4d2e3424d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:07.117430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d2103-1579-4603-84ef-1fd4d2e3424d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-fc2d2103-1579-4603-84ef-1fd4d2e3424d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:58:07.133847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:58:07.135157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:58:07.136228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:58:07.136228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:58:07.136228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:58:07.136228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:58:07.136228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:58:07.136228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:58:07.136228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:58:07.136228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:58:07.136228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:58:07.136228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:58:07.136228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 21 08:58:07.136228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:58:07.136228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 08:58:07.136228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:58:07.136228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 587/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jul 21 08:58:06 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:58:07.141846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acfc9e-f5f4-4c30-8905-30324b9e718c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:07.259153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acfc9e-f5f4-4c30-8905-30324b9e718c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:07.300025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419e161-2f74-4258-8127-6dca9f65a5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:07.300736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419e161-2f74-4258-8127-6dca9f65a5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:07.301048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419e161-2f74-4258-8127-6dca9f65a5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:07.301386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419e161-2f74-4258-8127-6dca9f65a5a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:07.302215 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419e161-2f74-4258-8127-6dca9f65a5a0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:58:07.302555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419e161-2f74-4258-8127-6dca9f65a5a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:07.304091 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419e161-2f74-4258-8127-6dca9f65a5a0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:07.312805 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 579/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jul 21 08:58:07 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:09.459094 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06ec1a-0adb-4693-a960-bf5628dec14f None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:09.459662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06ec1a-0adb-4693-a960-bf5628dec14f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:09.459662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06ec1a-0adb-4693-a960-bf5628dec14f None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:09.461073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 588/1167] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:58:09.466851 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff29849-aa95-4a9a-97c6-a3dc6d10dbf2 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:09.467388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff29849-aa95-4a9a-97c6-a3dc6d10dbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:09.467388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff29849-aa95-4a9a-97c6-a3dc6d10dbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:09.485080 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0ff29849-aa95-4a9a-97c6-a3dc6d10dbf2 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:58:09.486303 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0ff29849-aa95-4a9a-97c6-a3dc6d10dbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:58:09.519658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:09.559145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 580/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:58:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:58:09.584627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06ec1a-0adb-4693-a960-bf5628dec14f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:09.704928 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06ec1a-0adb-4693-a960-bf5628dec14f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:09.706518 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:09.742678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808f1a4-57ba-4326-9b6a-90e1ba417425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:09.743334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808f1a4-57ba-4326-9b6a-90e1ba417425 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:09.743613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808f1a4-57ba-4326-9b6a-90e1ba417425 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:09.743904 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808f1a4-57ba-4326-9b6a-90e1ba417425 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:09.744716 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808f1a4-57ba-4326-9b6a-90e1ba417425 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:09.746239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c808f1a4-57ba-4326-9b6a-90e1ba417425 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:09.762733 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 589/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Thu Jul 21 08:58:09 2022] POST /identity/v3/domains => generated 267 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:58:11.953738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3ff1a-7cbf-422a-8de6-8f3593f9641e None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:11.954286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3ff1a-7cbf-422a-8de6-8f3593f9641e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:11.954286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3ff1a-7cbf-422a-8de6-8f3593f9641e None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:11.955674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 581/1170] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:58:11.961184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af784bdb-7c60-4d2d-8801-f7be79156f1e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:11.961583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af784bdb-7c60-4d2d-8801-f7be79156f1e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:11.961736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af784bdb-7c60-4d2d-8801-f7be79156f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:11.978284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-af784bdb-7c60-4d2d-8801-f7be79156f1e None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:58:11.979383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-af784bdb-7c60-4d2d-8801-f7be79156f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:58:12.047507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 590/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:58:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:58:12.072379 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3ff1a-7cbf-422a-8de6-8f3593f9641e None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:12.175581 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3ff1a-7cbf-422a-8de6-8f3593f9641e None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:12.210118 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a9c06-5143-441a-97ab-e9cf44ef2208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:12.210999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a9c06-5143-441a-97ab-e9cf44ef2208 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:12.211285 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a9c06-5143-441a-97ab-e9cf44ef2208 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:12.211562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a9c06-5143-441a-97ab-e9cf44ef2208 None admin] PATH_INFO: `/v3/domains/7e115a78a3ea40e6a2f3c29faca7a683` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:12.222196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a9c06-5143-441a-97ab-e9cf44ef2208 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7e115a78a3ea40e6a2f3c29faca7a683)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:12.223665 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a9c06-5143-441a-97ab-e9cf44ef2208 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:12.225375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 582/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Thu Jul 21 08:58:12 2022] GET /identity/v3/domains/7e115a78a3ea40e6a2f3c29faca7a683 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:58:12.231352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af784bdb-7c60-4d2d-8801-f7be79156f1e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:12.321830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:12.331352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af784bdb-7c60-4d2d-8801-f7be79156f1e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:12.366803 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d784b-4868-4c08-aca0-cc15f4e33be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:12.367488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d784b-4868-4c08-aca0-cc15f4e33be7 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:12.367774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d784b-4868-4c08-aca0-cc15f4e33be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:12.368084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d784b-4868-4c08-aca0-cc15f4e33be7 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:12.368800 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d784b-4868-4c08-aca0-cc15f4e33be7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:12.370237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d784b-4868-4c08-aca0-cc15f4e33be7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:12.396337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 591/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jul 21 08:58:12 2022] POST /identity/v3/projects => generated 361 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:58:14.493496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895b731-91ca-429d-a1ae-66bb0ca9c4b1 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:14.494032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895b731-91ca-429d-a1ae-66bb0ca9c4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:14.494032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895b731-91ca-429d-a1ae-66bb0ca9c4b1 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:14.495406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 583/1174] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:58:14.500589 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96866659-ca7d-4713-bef4-2ff28c0ba079 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:14.501481 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96866659-ca7d-4713-bef4-2ff28c0ba079 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:14.501481 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96866659-ca7d-4713-bef4-2ff28c0ba079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:14.519046 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-96866659-ca7d-4713-bef4-2ff28c0ba079 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:58:14.520281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-96866659-ca7d-4713-bef4-2ff28c0ba079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:58:14.586040 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 592/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:58:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:58:14.611855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895b731-91ca-429d-a1ae-66bb0ca9c4b1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:14.642227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:14.716895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895b731-91ca-429d-a1ae-66bb0ca9c4b1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:14.752089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429d110-d077-4240-ac95-85a36ed07099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:14.752872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429d110-d077-4240-ac95-85a36ed07099 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:14.753160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429d110-d077-4240-ac95-85a36ed07099 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:14.753441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429d110-d077-4240-ac95-85a36ed07099 None admin] PATH_INFO: `/v3/domains/7e115a78a3ea40e6a2f3c29faca7a683` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:14.754631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429d110-d077-4240-ac95-85a36ed07099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7e115a78a3ea40e6a2f3c29faca7a683)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:14.756120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429d110-d077-4240-ac95-85a36ed07099 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:14.757756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 584/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Thu Jul 21 08:58:14 2022] GET /identity/v3/domains/7e115a78a3ea40e6a2f3c29faca7a683 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:58:14.763689 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96866659-ca7d-4713-bef4-2ff28c0ba079 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:14.836401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:14.861789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96866659-ca7d-4713-bef4-2ff28c0ba079 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:14.896521 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41618a3-c98b-4f70-a235-c044ffc945e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:14.897184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41618a3-c98b-4f70-a235-c044ffc945e4 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:14.897468 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41618a3-c98b-4f70-a235-c044ffc945e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:14.897743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41618a3-c98b-4f70-a235-c044ffc945e4 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:14.898454 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41618a3-c98b-4f70-a235-c044ffc945e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:14.899898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41618a3-c98b-4f70-a235-c044ffc945e4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:14.918480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 593/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:58:14 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:58:17.021185 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db826e3-07f3-4762-8c6c-577c6ec5e01d None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:17.021713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db826e3-07f3-4762-8c6c-577c6ec5e01d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:17.021713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db826e3-07f3-4762-8c6c-577c6ec5e01d None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:17.022993 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 585/1178] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:58:17.028415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cb0a2-9c6c-4ce6-ad2f-fca52803ed94 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:17.028867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cb0a2-9c6c-4ce6-ad2f-fca52803ed94 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:17.029008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cb0a2-9c6c-4ce6-ad2f-fca52803ed94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:17.046138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3a0cb0a2-9c6c-4ce6-ad2f-fca52803ed94 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:58:17.047325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3a0cb0a2-9c6c-4ce6-ad2f-fca52803ed94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:58:17.114264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 594/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:58:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:58:17.138446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db826e3-07f3-4762-8c6c-577c6ec5e01d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:17.237144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:17.245054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db826e3-07f3-4762-8c6c-577c6ec5e01d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:17.280300 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e63322-6588-4859-bbce-348063fb5dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:17.281002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e63322-6588-4859-bbce-348063fb5dab None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:17.281283 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e63322-6588-4859-bbce-348063fb5dab None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:17.281563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e63322-6588-4859-bbce-348063fb5dab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:17.292364 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e63322-6588-4859-bbce-348063fb5dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b3e63322-6588-4859-bbce-348063fb5dab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:58:17.301202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:58:17.302278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:58:17.303169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:58:17.303169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:58:17.303169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:58:17.303169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:58:17.303169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:58:17.303169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:58:17.303169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:58:17.303169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:58:17.303169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:58:17.303169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:58:17.303169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:58:17.303169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:58:17.303169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 586/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:58:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:58:17.308660 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0cb0a2-9c6c-4ce6-ad2f-fca52803ed94 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:17.372039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:17.421620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0cb0a2-9c6c-4ce6-ad2f-fca52803ed94 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:17.458940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208cebb-6d1c-4a77-9151-4a2c819690f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:17.459617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208cebb-6d1c-4a77-9151-4a2c819690f7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:17.459896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208cebb-6d1c-4a77-9151-4a2c819690f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:17.460197 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208cebb-6d1c-4a77-9151-4a2c819690f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:17.460955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208cebb-6d1c-4a77-9151-4a2c819690f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:58:17.461257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208cebb-6d1c-4a77-9151-4a2c819690f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:17.462649 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208cebb-6d1c-4a77-9151-4a2c819690f7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:17.471654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 595/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:58:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:17.477662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e63322-6588-4859-bbce-348063fb5dab None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:17.586565 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e63322-6588-4859-bbce-348063fb5dab None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:17.620449 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fb421-0011-4b2c-bb97-098ad307f8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:17.621316 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fb421-0011-4b2c-bb97-098ad307f8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:17.621601 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fb421-0011-4b2c-bb97-098ad307f8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:17.621875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fb421-0011-4b2c-bb97-098ad307f8b8 None admin] PATH_INFO: `/v3/users/c3b7e3a4736246c58c7439f513b90cdb` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:17.635553 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fb421-0011-4b2c-bb97-098ad307f8b8 None admin] RBAC: Authorizing `identity:get_user(user_id=c3b7e3a4736246c58c7439f513b90cdb)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:17.637103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fb421-0011-4b2c-bb97-098ad307f8b8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:17.638876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 587/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:58:17 2022] GET /identity/v3/users/c3b7e3a4736246c58c7439f513b90cdb => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:17.645375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208cebb-6d1c-4a77-9151-4a2c819690f7 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:17.752968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208cebb-6d1c-4a77-9151-4a2c819690f7 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:17.775515 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:17.788137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4eb90-2bfe-4933-851a-6ab167bf6d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:17.788787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4eb90-2bfe-4933-851a-6ab167bf6d36 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:17.789072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4eb90-2bfe-4933-851a-6ab167bf6d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:17.789351 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4eb90-2bfe-4933-851a-6ab167bf6d36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:17.856344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4eb90-2bfe-4933-851a-6ab167bf6d36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:17.857817 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4eb90-2bfe-4933-851a-6ab167bf6d36 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:17.958948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 596/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:58:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:58:17.964926 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fb421-0011-4b2c-bb97-098ad307f8b8 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:17.981177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:18.067386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fb421-0011-4b2c-bb97-098ad307f8b8 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:18.101751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605bd5f-f8c6-4a24-9f48-a90747409c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:18.102600 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605bd5f-f8c6-4a24-9f48-a90747409c1c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:18.102879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605bd5f-f8c6-4a24-9f48-a90747409c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:18.103151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605bd5f-f8c6-4a24-9f48-a90747409c1c None admin] PATH_INFO: `/v3/projects/7462fcbb55ab411ca35b634a00fb2185` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:18.104334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d605bd5f-f8c6-4a24-9f48-a90747409c1c None admin] RBAC: Authorizing `identity:get_project(project_id=7462fcbb55ab411ca35b634a00fb2185)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:18.105780 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d605bd5f-f8c6-4a24-9f48-a90747409c1c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:18.107529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 588/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:58:17 2022] GET /identity/v3/projects/7462fcbb55ab411ca35b634a00fb2185 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:18.113649 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4eb90-2bfe-4933-851a-6ab167bf6d36 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:18.136987 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:18.211728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4eb90-2bfe-4933-851a-6ab167bf6d36 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:18.244988 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e5900e-eea9-455b-be36-86865d6782af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:18.245625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5900e-eea9-455b-be36-86865d6782af None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:18.245903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5900e-eea9-455b-be36-86865d6782af None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:18.246173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e5900e-eea9-455b-be36-86865d6782af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:18.246997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5900e-eea9-455b-be36-86865d6782af None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=7462fcbb55ab411ca35b634a00fb2185, user.id=c3b7e3a4736246c58c7439f513b90cdb) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:58:18.247302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5900e-eea9-455b-be36-86865d6782af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:18.248753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e5900e-eea9-455b-be36-86865d6782af None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:18.254192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 597/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:58:18 2022] GET /identity/v3/role_assignments?user.id=c3b7e3a4736246c58c7439f513b90cdb&scope.project.id=7462fcbb55ab411ca35b634a00fb2185&role.id=000e942517f2469ba36d2e65978bc314 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:20.307967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357c7dd-e250-4078-8e2a-062d5d3abbe0 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:20.308503 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357c7dd-e250-4078-8e2a-062d5d3abbe0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:20.308503 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357c7dd-e250-4078-8e2a-062d5d3abbe0 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:20.309861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 589/1186] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:58:20.315200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98039467-3d8e-40fd-9ff9-2488c5726dcb None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:20.315636 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98039467-3d8e-40fd-9ff9-2488c5726dcb None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:20.315763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98039467-3d8e-40fd-9ff9-2488c5726dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:20.332926 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-98039467-3d8e-40fd-9ff9-2488c5726dcb None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:58:20.334121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-98039467-3d8e-40fd-9ff9-2488c5726dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:58:20.415432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 598/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:58:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:58:20.444696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357c7dd-e250-4078-8e2a-062d5d3abbe0 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:20.525916 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:20.545274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357c7dd-e250-4078-8e2a-062d5d3abbe0 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:20.579479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16e11f-b8cc-4406-b861-b3fe102ff62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:20.580302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16e11f-b8cc-4406-b861-b3fe102ff62d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:20.580580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16e11f-b8cc-4406-b861-b3fe102ff62d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:20.580852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16e11f-b8cc-4406-b861-b3fe102ff62d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:20.590629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16e11f-b8cc-4406-b861-b3fe102ff62d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-bf16e11f-b8cc-4406-b861-b3fe102ff62d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:58:20.598206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:58:20.599291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:58:20.600495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:58:20.600495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:58:20.600495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:58:20.600495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:58:20.600495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:58:20.600495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:58:20.600495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:58:20.600495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:58:20.600495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:58:20.600495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:58:20.600495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:58:20.600495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:58:20.600495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 590/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:58:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:58:20.605501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98039467-3d8e-40fd-9ff9-2488c5726dcb None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:20.623142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:20.706597 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98039467-3d8e-40fd-9ff9-2488c5726dcb None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:20.741419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a35df-708c-4638-a4c1-bcf98c74d862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:20.742077 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a35df-708c-4638-a4c1-bcf98c74d862 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:20.742356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a35df-708c-4638-a4c1-bcf98c74d862 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:20.742627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a35df-708c-4638-a4c1-bcf98c74d862 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:20.743376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a35df-708c-4638-a4c1-bcf98c74d862 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:58:20.743679 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a35df-708c-4638-a4c1-bcf98c74d862 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:20.745088 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a35df-708c-4638-a4c1-bcf98c74d862 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:20.753946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 599/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:58:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:20.760092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16e11f-b8cc-4406-b861-b3fe102ff62d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:20.859092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16e11f-b8cc-4406-b861-b3fe102ff62d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:20.892597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dcba9-9094-4a88-baf9-7c9315b6aa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:20.893448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dcba9-9094-4a88-baf9-7c9315b6aa09 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:20.893726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dcba9-9094-4a88-baf9-7c9315b6aa09 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:20.893996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dcba9-9094-4a88-baf9-7c9315b6aa09 None admin] PATH_INFO: `/v3/users/c3b7e3a4736246c58c7439f513b90cdb` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:20.895184 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2dcba9-9094-4a88-baf9-7c9315b6aa09 None admin] RBAC: Authorizing `identity:get_user(user_id=c3b7e3a4736246c58c7439f513b90cdb)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:20.896631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2dcba9-9094-4a88-baf9-7c9315b6aa09 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:20.898244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 591/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:58:20 2022] GET /identity/v3/users/c3b7e3a4736246c58c7439f513b90cdb => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:20.904534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a35df-708c-4638-a4c1-bcf98c74d862 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:20.983555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:21.005761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a35df-708c-4638-a4c1-bcf98c74d862 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:21.037882 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502b111-4bf8-4d5d-b85d-3bf5dfcfe061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:21.038733 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502b111-4bf8-4d5d-b85d-3bf5dfcfe061 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:21.039018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502b111-4bf8-4d5d-b85d-3bf5dfcfe061 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:21.039290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502b111-4bf8-4d5d-b85d-3bf5dfcfe061 None admin] PATH_INFO: `/v3/projects/7462fcbb55ab411ca35b634a00fb2185` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:21.040479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502b111-4bf8-4d5d-b85d-3bf5dfcfe061 None admin] RBAC: Authorizing `identity:get_project(project_id=7462fcbb55ab411ca35b634a00fb2185)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:21.041925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502b111-4bf8-4d5d-b85d-3bf5dfcfe061 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:21.043609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 600/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:58:20 2022] GET /identity/v3/projects/7462fcbb55ab411ca35b634a00fb2185 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:21.050522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dcba9-9094-4a88-baf9-7c9315b6aa09 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:21.063131 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:21.154254 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dcba9-9094-4a88-baf9-7c9315b6aa09 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:21.187407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f086178-0781-4671-9e4b-d87a5f2b167b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:21.188851 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f086178-0781-4671-9e4b-d87a5f2b167b None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:21.189364 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f086178-0781-4671-9e4b-d87a5f2b167b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:21.189475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f086178-0781-4671-9e4b-d87a5f2b167b None admin] PATH_INFO: `/v3/projects/7462fcbb55ab411ca35b634a00fb2185/users/c3b7e3a4736246c58c7439f513b90cdb/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:21.191377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f086178-0781-4671-9e4b-d87a5f2b167b None admin] RBAC: Authorizing `identity:create_grant(project_id=7462fcbb55ab411ca35b634a00fb2185, user_id=c3b7e3a4736246c58c7439f513b90cdb, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:21.192857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f086178-0781-4671-9e4b-d87a5f2b167b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:21.205539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 592/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:58:21 2022] PUT /identity/v3/projects/7462fcbb55ab411ca35b634a00fb2185/users/c3b7e3a4736246c58c7439f513b90cdb/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:58:23.228721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a6628-0582-4204-be1d-078e42a4fb72 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:23.229303 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a6628-0582-4204-be1d-078e42a4fb72 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:23.229303 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a6628-0582-4204-be1d-078e42a4fb72 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:23.230614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 601/1193] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:58:23.235789 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0d335-6ba1-476f-ab13-d49e0eb848aa None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:23.236255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0d335-6ba1-476f-ab13-d49e0eb848aa None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:23.236386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0d335-6ba1-476f-ab13-d49e0eb848aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:23.253126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c9b0d335-6ba1-476f-ab13-d49e0eb848aa None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:58:23.254308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c9b0d335-6ba1-476f-ab13-d49e0eb848aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:58:23.315355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:23.320237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 593/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:58:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:58:23.344377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a6628-0582-4204-be1d-078e42a4fb72 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:23.446110 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a6628-0582-4204-be1d-078e42a4fb72 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:23.479104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e02ee-c1a0-408d-860f-82b117072b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:23.479804 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e02ee-c1a0-408d-860f-82b117072b99 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:23.480115 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e02ee-c1a0-408d-860f-82b117072b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:23.480389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e02ee-c1a0-408d-860f-82b117072b99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:23.491401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8e02ee-c1a0-408d-860f-82b117072b99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-7e8e02ee-c1a0-408d-860f-82b117072b99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:58:23.499096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:58:23.500395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:58:23.501356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:58:23.501356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:58:23.501356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:58:23.501356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:58:23.501356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:58:23.501356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:58:23.501356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:58:23.501356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:58:23.501356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:58:23.501356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:58:23.501356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:58:23.501356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:58:23.501356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 602/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:58:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:58:23.507483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0d335-6ba1-476f-ab13-d49e0eb848aa None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:23.607270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0d335-6ba1-476f-ab13-d49e0eb848aa None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:23.640922 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd665ec-441e-4c40-9b06-72f04f517807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:23.642134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd665ec-441e-4c40-9b06-72f04f517807 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:23.642419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd665ec-441e-4c40-9b06-72f04f517807 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:23.642689 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd665ec-441e-4c40-9b06-72f04f517807 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:23.643426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd665ec-441e-4c40-9b06-72f04f517807 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:58:23.643732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd665ec-441e-4c40-9b06-72f04f517807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:23.645156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd665ec-441e-4c40-9b06-72f04f517807 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:23.653353 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 594/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:58:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:23.659407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e02ee-c1a0-408d-860f-82b117072b99 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:23.668418 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:23.761598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e02ee-c1a0-408d-860f-82b117072b99 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:23.795146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db545cbd-cfd3-4c4c-8aa1-346ab6f40850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:23.796058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db545cbd-cfd3-4c4c-8aa1-346ab6f40850 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:23.796340 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db545cbd-cfd3-4c4c-8aa1-346ab6f40850 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:23.796614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db545cbd-cfd3-4c4c-8aa1-346ab6f40850 None admin] PATH_INFO: `/v3/users/c3b7e3a4736246c58c7439f513b90cdb` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:23.798674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db545cbd-cfd3-4c4c-8aa1-346ab6f40850 None admin] RBAC: Authorizing `identity:get_user(user_id=c3b7e3a4736246c58c7439f513b90cdb)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:23.800485 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db545cbd-cfd3-4c4c-8aa1-346ab6f40850 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:23.802108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 603/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jul 21 08:58:23 2022] GET /identity/v3/users/c3b7e3a4736246c58c7439f513b90cdb => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:23.807971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd665ec-441e-4c40-9b06-72f04f517807 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:23.868832 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:23.906870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd665ec-441e-4c40-9b06-72f04f517807 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:23.940185 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c3f8c-221a-49fb-ace8-958d787ca898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:23.940821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c3f8c-221a-49fb-ace8-958d787ca898 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:23.941109 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c3f8c-221a-49fb-ace8-958d787ca898 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:23.941385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c3f8c-221a-49fb-ace8-958d787ca898 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:24.007415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c3f8c-221a-49fb-ace8-958d787ca898 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:24.008913 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c3f8c-221a-49fb-ace8-958d787ca898 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:24.068437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:24.109073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 595/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:58:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:58:24.115162 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db545cbd-cfd3-4c4c-8aa1-346ab6f40850 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:24.187939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:24.212644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db545cbd-cfd3-4c4c-8aa1-346ab6f40850 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:24.245865 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db637ee0-78ee-4b15-8b4f-4ed8005475c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:24.246715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db637ee0-78ee-4b15-8b4f-4ed8005475c4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:24.246996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db637ee0-78ee-4b15-8b4f-4ed8005475c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:24.247267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db637ee0-78ee-4b15-8b4f-4ed8005475c4 None admin] PATH_INFO: `/v3/projects/7462fcbb55ab411ca35b634a00fb2185` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:24.248448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db637ee0-78ee-4b15-8b4f-4ed8005475c4 None admin] RBAC: Authorizing `identity:get_project(project_id=7462fcbb55ab411ca35b634a00fb2185)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:24.249887 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db637ee0-78ee-4b15-8b4f-4ed8005475c4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:24.251588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 604/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jul 21 08:58:24 2022] GET /identity/v3/projects/7462fcbb55ab411ca35b634a00fb2185 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:24.258011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c3f8c-221a-49fb-ace8-958d787ca898 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:58:24.359148 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c3f8c-221a-49fb-ace8-958d787ca898 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:58:24.394499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3c182-cf98-412d-b79d-abcacb5a21f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:58:24.395256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3c182-cf98-412d-b79d-abcacb5a21f6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:58:24.395541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3c182-cf98-412d-b79d-abcacb5a21f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:58:24.395826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3c182-cf98-412d-b79d-abcacb5a21f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:58:24.396822 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3c182-cf98-412d-b79d-abcacb5a21f6 None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=7462fcbb55ab411ca35b634a00fb2185, user.id=c3b7e3a4736246c58c7439f513b90cdb) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:58:24.397174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3c182-cf98-412d-b79d-abcacb5a21f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:58:24.398812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3c182-cf98-412d-b79d-abcacb5a21f6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:58:24.406104 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 596/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:58:24 2022] GET /identity/v3/role_assignments?user.id=c3b7e3a4736246c58c7439f513b90cdb&scope.project.id=7462fcbb55ab411ca35b634a00fb2185&role.id=000e942517f2469ba36d2e65978bc314 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:58:34.242006 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=29 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:58:34.402633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=87 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:22.433361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeee340-bd0f-4b63-9de2-8c29675e50dc None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:22.433980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeee340-bd0f-4b63-9de2-8c29675e50dc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:22.433980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeee340-bd0f-4b63-9de2-8c29675e50dc None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:22.435392 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 605/1201] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:59:22.442388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823f3a1-ac3d-419c-9e51-d5a1eadf4ee2 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:22.442939 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823f3a1-ac3d-419c-9e51-d5a1eadf4ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:22.443122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823f3a1-ac3d-419c-9e51-d5a1eadf4ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:22.464759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d823f3a1-ac3d-419c-9e51-d5a1eadf4ee2 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:59:22.465978 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d823f3a1-ac3d-419c-9e51-d5a1eadf4ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:59:22.568213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 597/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:59:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:59:22.597166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeee340-bd0f-4b63-9de2-8c29675e50dc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:22.745308 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeee340-bd0f-4b63-9de2-8c29675e50dc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:22.802285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef9558-298f-4908-be5c-59dc9f9de8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:22.803140 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef9558-298f-4908-be5c-59dc9f9de8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:22.803442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef9558-298f-4908-be5c-59dc9f9de8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:22.803744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef9558-298f-4908-be5c-59dc9f9de8d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:22.820934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef9558-298f-4908-be5c-59dc9f9de8d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:22.823116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef9558-298f-4908-be5c-59dc9f9de8d1 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:22.860975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-39ef9558-298f-4908-be5c-59dc9f9de8d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:22.860975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:22.860975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:22.860975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:22.860975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:22.860975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:22.860975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:22.860975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:22.860975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:22.860975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:22.860975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:22.860975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:22.860975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:59:22.860975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:59:22.860975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:22.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:22.863251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:59:22.863251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:59:22.863251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:22.863251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:22.863251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 606/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:22.872831 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823f3a1-ac3d-419c-9e51-d5a1eadf4ee2 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:23.041684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823f3a1-ac3d-419c-9e51-d5a1eadf4ee2 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:23.082402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:23.091808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e82f9-6efc-4129-a259-5f1220e65573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:23.093106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e82f9-6efc-4129-a259-5f1220e65573 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:23.093106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e82f9-6efc-4129-a259-5f1220e65573 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:23.093336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e82f9-6efc-4129-a259-5f1220e65573 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:23.094017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e82f9-6efc-4129-a259-5f1220e65573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:23.094425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e82f9-6efc-4129-a259-5f1220e65573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:23.096153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e82f9-6efc-4129-a259-5f1220e65573 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:23.120264 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 598/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:23.122135 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef9558-298f-4908-be5c-59dc9f9de8d1 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:23.207490 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:23.273631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef9558-298f-4908-be5c-59dc9f9de8d1 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:23.316004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f81ee5-b72c-4ae7-8d5a-f299cef54caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:23.316799 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f81ee5-b72c-4ae7-8d5a-f299cef54caf None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:23.317217 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f81ee5-b72c-4ae7-8d5a-f299cef54caf None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:23.317594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f81ee5-b72c-4ae7-8d5a-f299cef54caf None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:23.318490 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f81ee5-b72c-4ae7-8d5a-f299cef54caf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:23.320135 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f81ee5-b72c-4ae7-8d5a-f299cef54caf None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:23.342409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 607/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jul 21 08:59:23 2022] POST /identity/v3/users => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:59:25.808060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424111a0-63bc-44a1-94dd-a8ac725cd687 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:25.808060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424111a0-63bc-44a1-94dd-a8ac725cd687 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:25.808060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424111a0-63bc-44a1-94dd-a8ac725cd687 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:25.808844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 599/1206] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:59:25.818540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca435b-9a4b-4268-b159-22c385ea468e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:25.819110 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca435b-9a4b-4268-b159-22c385ea468e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:25.819531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca435b-9a4b-4268-b159-22c385ea468e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:25.848980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-49ca435b-9a4b-4268-b159-22c385ea468e None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:59:25.850633 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-49ca435b-9a4b-4268-b159-22c385ea468e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:59:25.947332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 608/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:59:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:59:25.974696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424111a0-63bc-44a1-94dd-a8ac725cd687 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:26.127707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424111a0-63bc-44a1-94dd-a8ac725cd687 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:26.179222 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03292b21-e24d-4393-8f99-c4bb667192f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:26.180117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03292b21-e24d-4393-8f99-c4bb667192f7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:26.180438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03292b21-e24d-4393-8f99-c4bb667192f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:26.180715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03292b21-e24d-4393-8f99-c4bb667192f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:26.193076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03292b21-e24d-4393-8f99-c4bb667192f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-03292b21-e24d-4393-8f99-c4bb667192f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:59:26.201195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:26.210538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:26.212242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:26.212242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:26.212242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:26.212242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:26.212242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:59:26.212242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:59:26.212242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:59:26.212242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:59:26.212242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:59:26.212242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:59:26.212242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:59:26.212242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:26.212242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 600/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:59:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:26.213541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca435b-9a4b-4268-b159-22c385ea468e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:26.257548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:26.341549 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca435b-9a4b-4268-b159-22c385ea468e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:26.401379 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c9ce6-58b0-48c2-9dfb-f14282c9113f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:26.402218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c9ce6-58b0-48c2-9dfb-f14282c9113f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:26.402551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c9ce6-58b0-48c2-9dfb-f14282c9113f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:26.402832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c9ce6-58b0-48c2-9dfb-f14282c9113f None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:26.403730 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c9ce6-58b0-48c2-9dfb-f14282c9113f None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:26.404148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c9ce6-58b0-48c2-9dfb-f14282c9113f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:26.405642 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c9ce6-58b0-48c2-9dfb-f14282c9113f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:26.417166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 609/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:59:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:59:26.424915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03292b21-e24d-4393-8f99-c4bb667192f7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:26.505908 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:26.589999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03292b21-e24d-4393-8f99-c4bb667192f7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:26.644262 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ceeb9-b969-45cc-8221-e7bc1560a280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:26.645343 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ceeb9-b969-45cc-8221-e7bc1560a280 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:26.645866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ceeb9-b969-45cc-8221-e7bc1560a280 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:26.646144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ceeb9-b969-45cc-8221-e7bc1560a280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:26.661138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ceeb9-b969-45cc-8221-e7bc1560a280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:26.662648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ceeb9-b969-45cc-8221-e7bc1560a280 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:26.683253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-da3ceeb9-b969-45cc-8221-e7bc1560a280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:26.683253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:26.683253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:26.683253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:26.683253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:26.683253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:26.683253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:26.683253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:26.683253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:26.683253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:26.683253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:26.683253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:26.683253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:59:26.683253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:59:26.683253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:26.685171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:26.687024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:59:26.687024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:59:26.687024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:26.687024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:26.690079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 601/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:26.691025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c9ce6-58b0-48c2-9dfb-f14282c9113f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:26.826251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:26.829907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c9ce6-58b0-48c2-9dfb-f14282c9113f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:26.866004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffe931-ae29-4512-abe8-1b3f34901234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:26.866587 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffe931-ae29-4512-abe8-1b3f34901234 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:26.866870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffe931-ae29-4512-abe8-1b3f34901234 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:26.867154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffe931-ae29-4512-abe8-1b3f34901234 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:26.867939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffe931-ae29-4512-abe8-1b3f34901234 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:26.868262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffe931-ae29-4512-abe8-1b3f34901234 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:26.869794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffe931-ae29-4512-abe8-1b3f34901234 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:26.885765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 610/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:59:26.891993 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ceeb9-b969-45cc-8221-e7bc1560a280 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:27.011569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ceeb9-b969-45cc-8221-e7bc1560a280 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:27.014246 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:27.066692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d7090-e361-4e21-90dc-09bfc9f49aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:27.067588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d7090-e361-4e21-90dc-09bfc9f49aea None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:27.067986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d7090-e361-4e21-90dc-09bfc9f49aea None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:27.068211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d7090-e361-4e21-90dc-09bfc9f49aea None admin] PATH_INFO: `/v3/users/placement` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:27.083166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d7090-e361-4e21-90dc-09bfc9f49aea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:27.084811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d7090-e361-4e21-90dc-09bfc9f49aea None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-c20d7090-e361-4e21-90dc-09bfc9f49aea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:59:27.102331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:27.103406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:27.109268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:27.109268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:27.109268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:59:27.109268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:59:27.109268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:59:27.109268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:59:27.109268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:59:27.109268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:59:27.109268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:59:27.109268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:27.109268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 602/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:27.109935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffe931-ae29-4512-abe8-1b3f34901234 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:27.242948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffe931-ae29-4512-abe8-1b3f34901234 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:27.288719 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51a5c5-6e8c-4af0-b43b-bec396672bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:27.289478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51a5c5-6e8c-4af0-b43b-bec396672bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:27.289707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51a5c5-6e8c-4af0-b43b-bec396672bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:27.289990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51a5c5-6e8c-4af0-b43b-bec396672bc8 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:27.291969 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51a5c5-6e8c-4af0-b43b-bec396672bc8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:27.291969 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51a5c5-6e8c-4af0-b43b-bec396672bc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:27.292745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51a5c5-6e8c-4af0-b43b-bec396672bc8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:27.315288 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 611/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jul 21 08:59:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:27.321375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d7090-e361-4e21-90dc-09bfc9f49aea None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:27.462628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d7090-e361-4e21-90dc-09bfc9f49aea None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:27.515396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fbc19-68eb-4a92-a64d-1ce195513f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:27.516035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fbc19-68eb-4a92-a64d-1ce195513f4d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:27.516930 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fbc19-68eb-4a92-a64d-1ce195513f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:27.517371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fbc19-68eb-4a92-a64d-1ce195513f4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:27.567162 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:27.618759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fbc19-68eb-4a92-a64d-1ce195513f4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:27.620478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fbc19-68eb-4a92-a64d-1ce195513f4d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:27.771978 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 603/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:59:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 08:59:27.779770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51a5c5-6e8c-4af0-b43b-bec396672bc8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:27.807299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:27.963423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51a5c5-6e8c-4af0-b43b-bec396672bc8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:28.027096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d570c0d-5960-4fa9-921d-4a6a8e21d074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:28.028735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d570c0d-5960-4fa9-921d-4a6a8e21d074 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:28.029034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d570c0d-5960-4fa9-921d-4a6a8e21d074 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:28.029293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d570c0d-5960-4fa9-921d-4a6a8e21d074 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:28.044005 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d570c0d-5960-4fa9-921d-4a6a8e21d074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:28.045695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d570c0d-5960-4fa9-921d-4a6a8e21d074 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:28.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-4d570c0d-5960-4fa9-921d-4a6a8e21d074 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:28.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:28.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:28.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:28.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:28.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:28.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:28.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:28.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:28.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:28.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:28.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:28.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:59:28.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:59:28.069304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:28.071073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:28.075189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:59:28.075189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:59:28.075189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:28.075189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:28.075189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 612/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:28.076993 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fbc19-68eb-4a92-a64d-1ce195513f4d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:28.166686 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:28.215588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fbc19-68eb-4a92-a64d-1ce195513f4d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:28.272136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1025f5-88b0-47e9-9649-5a32c5e30963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:28.272852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1025f5-88b0-47e9-9649-5a32c5e30963 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:28.273127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1025f5-88b0-47e9-9649-5a32c5e30963 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:28.273397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1025f5-88b0-47e9-9649-5a32c5e30963 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:28.274147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1025f5-88b0-47e9-9649-5a32c5e30963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:28.274455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1025f5-88b0-47e9-9649-5a32c5e30963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:28.275873 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1025f5-88b0-47e9-9649-5a32c5e30963 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:28.302129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 604/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:28.309343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d570c0d-5960-4fa9-921d-4a6a8e21d074 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:28.416293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:28.457513 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d570c0d-5960-4fa9-921d-4a6a8e21d074 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:28.502115 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400f6d8-2784-4a0b-b821-b98b16d2ee22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:28.502996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400f6d8-2784-4a0b-b821-b98b16d2ee22 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:28.503311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400f6d8-2784-4a0b-b821-b98b16d2ee22 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:28.503586 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400f6d8-2784-4a0b-b821-b98b16d2ee22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:28.512923 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400f6d8-2784-4a0b-b821-b98b16d2ee22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:28.514435 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400f6d8-2784-4a0b-b821-b98b16d2ee22 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:28.541259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-a400f6d8-2784-4a0b-b821-b98b16d2ee22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:59:28.541259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:28.541259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:28.541259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:28.541259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:28.541259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:28.541259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:28.541259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:28.541259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:28.541259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:28.541259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:28.541259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:28.541259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:59:28.541259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:59:28.541259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:28.542427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:28.543551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:59:28.543551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:59:28.543551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:59:28.543551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:59:28.543551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:59:28.543551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:59:28.543551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:59:28.543551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:28.544045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 613/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:59:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:59:28.554589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1025f5-88b0-47e9-9649-5a32c5e30963 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:28.694362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1025f5-88b0-47e9-9649-5a32c5e30963 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:28.725692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:28.744456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1438477-1ece-45e7-9e15-e555cdb1d6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:28.745136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1438477-1ece-45e7-9e15-e555cdb1d6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:28.745423 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1438477-1ece-45e7-9e15-e555cdb1d6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:28.745700 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1438477-1ece-45e7-9e15-e555cdb1d6f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:28.746606 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1438477-1ece-45e7-9e15-e555cdb1d6f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:28.746950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1438477-1ece-45e7-9e15-e555cdb1d6f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:28.748759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1438477-1ece-45e7-9e15-e555cdb1d6f2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:28.765465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 605/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:59:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:28.772756 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400f6d8-2784-4a0b-b821-b98b16d2ee22 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:28.903073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400f6d8-2784-4a0b-b821-b98b16d2ee22 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:28.945258 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:28.956176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436fe051-2928-4c14-8e63-2fdf834b6dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:28.957280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fe051-2928-4c14-8e63-2fdf834b6dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:28.957807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fe051-2928-4c14-8e63-2fdf834b6dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:28.958294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fe051-2928-4c14-8e63-2fdf834b6dd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:28.959847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436fe051-2928-4c14-8e63-2fdf834b6dd0 None admin] RBAC: Adding query filter params (role.id=8511cf3e5d2c4acdae6d0ffda5e72c63, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=72f9ef6de66e456e800237e9f954e98d) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:28.960441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436fe051-2928-4c14-8e63-2fdf834b6dd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:28.963127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436fe051-2928-4c14-8e63-2fdf834b6dd0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:28.975257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 614/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:59:28 2022] GET /identity/v3/role_assignments?user.id=72f9ef6de66e456e800237e9f954e98d&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:59:31.353662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18fdf5-1bab-4b0d-84a2-11c58a01d3dd None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:31.354541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18fdf5-1bab-4b0d-84a2-11c58a01d3dd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:31.354541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18fdf5-1bab-4b0d-84a2-11c58a01d3dd None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:31.356175 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 606/1220] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:59:31.367690 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a436c88-188b-4802-ab4c-b0fa6f297312 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:31.367690 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a436c88-188b-4802-ab4c-b0fa6f297312 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:31.367690 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a436c88-188b-4802-ab4c-b0fa6f297312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:31.393717 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-9a436c88-188b-4802-ab4c-b0fa6f297312 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:59:31.395007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9a436c88-188b-4802-ab4c-b0fa6f297312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:59:31.477181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 615/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:59:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:59:31.501883 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18fdf5-1bab-4b0d-84a2-11c58a01d3dd None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:31.685067 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18fdf5-1bab-4b0d-84a2-11c58a01d3dd None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:31.741851 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d58ac2-b40d-4840-9597-d784bca4cd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:31.743242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d58ac2-b40d-4840-9597-d784bca4cd13 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:31.743728 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d58ac2-b40d-4840-9597-d784bca4cd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:31.744273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d58ac2-b40d-4840-9597-d784bca4cd13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:31.759905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d58ac2-b40d-4840-9597-d784bca4cd13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-49d58ac2-b40d-4840-9597-d784bca4cd13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:59:31.776043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:31.777144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:31.778003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:31.778003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:31.778003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:31.778003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:31.778003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:59:31.778003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:59:31.778003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:59:31.778003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:59:31.778003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:59:31.778003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:59:31.778003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:59:31.778003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:31.778846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 607/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:59:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:31.789283 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a436c88-188b-4802-ab4c-b0fa6f297312 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:31.973879 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a436c88-188b-4802-ab4c-b0fa6f297312 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:32.021754 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:32.042421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef06b8-b0c1-4d59-b804-802f3d830769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:32.043615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef06b8-b0c1-4d59-b804-802f3d830769 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:32.044175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef06b8-b0c1-4d59-b804-802f3d830769 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:32.044682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef06b8-b0c1-4d59-b804-802f3d830769 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:32.046138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef06b8-b0c1-4d59-b804-802f3d830769 None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:32.046705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef06b8-b0c1-4d59-b804-802f3d830769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:32.050055 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef06b8-b0c1-4d59-b804-802f3d830769 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:32.067065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 616/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:59:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:32.072734 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d58ac2-b40d-4840-9597-d784bca4cd13 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:32.138723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:32.239800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d58ac2-b40d-4840-9597-d784bca4cd13 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:32.292255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc2ee8-ac21-4a32-89d7-bb8bc60cf32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:32.293307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc2ee8-ac21-4a32-89d7-bb8bc60cf32b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:32.293727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc2ee8-ac21-4a32-89d7-bb8bc60cf32b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:32.294158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc2ee8-ac21-4a32-89d7-bb8bc60cf32b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:32.309396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc2ee8-ac21-4a32-89d7-bb8bc60cf32b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:32.311193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc2ee8-ac21-4a32-89d7-bb8bc60cf32b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:32.332811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-fffc2ee8-ac21-4a32-89d7-bb8bc60cf32b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:32.332811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:32.332811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:32.332811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:32.332811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:32.332811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:32.332811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:32.332811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:32.332811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:32.332811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:32.332811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:32.332811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:32.332811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:59:32.332811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:59:32.332811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:32.334876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:32.336650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:59:32.336650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:59:32.336650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:32.336650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:32.336650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 608/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:32.345056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef06b8-b0c1-4d59-b804-802f3d830769 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:32.501621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef06b8-b0c1-4d59-b804-802f3d830769 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:32.551587 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b2ae0-9f58-4826-bad8-647168e47245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:32.552837 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b2ae0-9f58-4826-bad8-647168e47245 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:32.553319 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b2ae0-9f58-4826-bad8-647168e47245 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:32.553696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b2ae0-9f58-4826-bad8-647168e47245 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:32.554465 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b2ae0-9f58-4826-bad8-647168e47245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:32.554772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b2ae0-9f58-4826-bad8-647168e47245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:32.556202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b2ae0-9f58-4826-bad8-647168e47245 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:32.569263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 617/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:32.576606 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc2ee8-ac21-4a32-89d7-bb8bc60cf32b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:32.744051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:32.755153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc2ee8-ac21-4a32-89d7-bb8bc60cf32b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:32.805922 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5167f12-9008-4ab3-b770-ad8a66b7b36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:32.806753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5167f12-9008-4ab3-b770-ad8a66b7b36f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:32.807043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5167f12-9008-4ab3-b770-ad8a66b7b36f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:32.807328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5167f12-9008-4ab3-b770-ad8a66b7b36f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:32.818558 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5167f12-9008-4ab3-b770-ad8a66b7b36f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:32.820286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5167f12-9008-4ab3-b770-ad8a66b7b36f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-a5167f12-9008-4ab3-b770-ad8a66b7b36f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:59:32.838809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:32.840236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:32.841544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:32.841544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:32.841544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:59:32.841544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:59:32.841544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:59:32.841544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:59:32.841544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:59:32.841544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:59:32.841544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:59:32.841544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:32.841544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 609/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:32.848230 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b2ae0-9f58-4826-bad8-647168e47245 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:32.936661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:33.025366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b2ae0-9f58-4826-bad8-647168e47245 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:33.073604 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178648d-2049-457c-9514-c121717c4f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:33.074543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178648d-2049-457c-9514-c121717c4f06 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:33.074982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178648d-2049-457c-9514-c121717c4f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:33.075414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178648d-2049-457c-9514-c121717c4f06 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:33.076760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178648d-2049-457c-9514-c121717c4f06 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:33.077102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178648d-2049-457c-9514-c121717c4f06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:33.079754 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178648d-2049-457c-9514-c121717c4f06 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:33.113108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 618/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jul 21 08:59:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:33.114473 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5167f12-9008-4ab3-b770-ad8a66b7b36f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:33.267402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5167f12-9008-4ab3-b770-ad8a66b7b36f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:33.312623 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84698c6e-192a-489f-9b84-c7f61ce2facd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:33.314110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84698c6e-192a-489f-9b84-c7f61ce2facd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:33.314477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84698c6e-192a-489f-9b84-c7f61ce2facd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:33.314877 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84698c6e-192a-489f-9b84-c7f61ce2facd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:33.332727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84698c6e-192a-489f-9b84-c7f61ce2facd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:33.334975 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84698c6e-192a-489f-9b84-c7f61ce2facd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:33.361915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-84698c6e-192a-489f-9b84-c7f61ce2facd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:33.361915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:33.361915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:33.361915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:33.361915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:33.361915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:33.361915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:33.361915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:33.361915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:33.361915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:33.361915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:33.361915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:33.361915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:59:33.361915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:59:33.361915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:33.370675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:33.373495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:59:33.373495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:59:33.373495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:33.373495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:33.373495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 610/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:59:33.374199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178648d-2049-457c-9514-c121717c4f06 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:33.559900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178648d-2049-457c-9514-c121717c4f06 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:33.564120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:33.616880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d24049-faff-4c41-a31a-1fe28fccc8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:33.617689 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d24049-faff-4c41-a31a-1fe28fccc8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:33.618078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d24049-faff-4c41-a31a-1fe28fccc8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:33.618522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d24049-faff-4c41-a31a-1fe28fccc8a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:33.619622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d24049-faff-4c41-a31a-1fe28fccc8a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:33.620056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d24049-faff-4c41-a31a-1fe28fccc8a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:33.621634 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d24049-faff-4c41-a31a-1fe28fccc8a9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:33.640176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 619/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:59:33.649127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84698c6e-192a-489f-9b84-c7f61ce2facd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:33.662342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:33.820862 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84698c6e-192a-489f-9b84-c7f61ce2facd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:33.876808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0803f54-7c98-4c90-94f4-fec188d928a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:33.878151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0803f54-7c98-4c90-94f4-fec188d928a4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:33.878575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0803f54-7c98-4c90-94f4-fec188d928a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:33.879013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0803f54-7c98-4c90-94f4-fec188d928a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:33.891933 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0803f54-7c98-4c90-94f4-fec188d928a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:33.895203 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0803f54-7c98-4c90-94f4-fec188d928a4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:33.927685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-c0803f54-7c98-4c90-94f4-fec188d928a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:59:33.927685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:33.927685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:33.927685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:33.927685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:33.927685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:33.927685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:33.927685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:33.927685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:33.927685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:33.927685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:33.927685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:33.927685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:59:33.927685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:59:33.927685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:33.929733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:33.931741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:59:33.931741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:59:33.931741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:59:33.931741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:59:33.931741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:59:33.931741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:59:33.931741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:59:33.931741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:33.931741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 611/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:59:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:33.939514 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d24049-faff-4c41-a31a-1fe28fccc8a9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:34.097151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d24049-faff-4c41-a31a-1fe28fccc8a9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:34.147375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf09f0b-36d5-4120-a56e-e4a6ef277e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:34.148445 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf09f0b-36d5-4120-a56e-e4a6ef277e0c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:34.148944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf09f0b-36d5-4120-a56e-e4a6ef277e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:34.149348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf09f0b-36d5-4120-a56e-e4a6ef277e0c None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:34.151152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf09f0b-36d5-4120-a56e-e4a6ef277e0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:34.152772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf09f0b-36d5-4120-a56e-e4a6ef277e0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:34.154279 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf09f0b-36d5-4120-a56e-e4a6ef277e0c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:34.175782 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 620/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:59:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:34.186797 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0803f54-7c98-4c90-94f4-fec188d928a4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:34.307635 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:34.371488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0803f54-7c98-4c90-94f4-fec188d928a4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:34.423573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e109108-330c-4112-8a2d-f0f5315173c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:34.424833 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e109108-330c-4112-8a2d-f0f5315173c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:34.425204 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e109108-330c-4112-8a2d-f0f5315173c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:34.425519 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e109108-330c-4112-8a2d-f0f5315173c3 None admin] PATH_INFO: `/v3/projects/7fe0f489cbce43859f050c3270e00709/users/72f9ef6de66e456e800237e9f954e98d/roles/8511cf3e5d2c4acdae6d0ffda5e72c63` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:34.449857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e109108-330c-4112-8a2d-f0f5315173c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fe0f489cbce43859f050c3270e00709, user_id=72f9ef6de66e456e800237e9f954e98d, role_id=8511cf3e5d2c4acdae6d0ffda5e72c63)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:34.451560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e109108-330c-4112-8a2d-f0f5315173c3 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:34.479580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 612/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:59:34 2022] PUT /identity/v3/projects/7fe0f489cbce43859f050c3270e00709/users/72f9ef6de66e456e800237e9f954e98d/roles/8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:59:36.947409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78f43b-cc45-4e66-b910-6086146934c0 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:36.948411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78f43b-cc45-4e66-b910-6086146934c0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:36.948411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78f43b-cc45-4e66-b910-6086146934c0 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:36.950031 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 621/1233] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:59:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:59:36.959569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f750164-ca6d-4541-a9b7-5d40ca307070 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:36.960226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f750164-ca6d-4541-a9b7-5d40ca307070 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:36.960560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f750164-ca6d-4541-a9b7-5d40ca307070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:36.983404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2f750164-ca6d-4541-a9b7-5d40ca307070 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:59:36.985129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2f750164-ca6d-4541-a9b7-5d40ca307070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:59:37.082524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 613/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:59:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:59:37.108049 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78f43b-cc45-4e66-b910-6086146934c0 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:37.141891 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:37.264402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78f43b-cc45-4e66-b910-6086146934c0 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:37.339618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52609fcf-1c3d-4fff-b6ef-4d3efac7d240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:37.340490 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52609fcf-1c3d-4fff-b6ef-4d3efac7d240 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:37.340776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52609fcf-1c3d-4fff-b6ef-4d3efac7d240 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:37.341061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52609fcf-1c3d-4fff-b6ef-4d3efac7d240 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:37.354538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52609fcf-1c3d-4fff-b6ef-4d3efac7d240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-52609fcf-1c3d-4fff-b6ef-4d3efac7d240 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:59:37.364684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:37.376128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:37.378072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:37.378072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:37.378072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:37.378072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:37.378072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:59:37.378072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:59:37.378072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:59:37.378072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:59:37.378072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:59:37.378072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:59:37.378072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:59:37.378072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:37.378072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 622/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jul 21 08:59:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:37.381480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f750164-ca6d-4541-a9b7-5d40ca307070 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:37.468735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:37.533412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f750164-ca6d-4541-a9b7-5d40ca307070 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:37.588529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46056af0-d29b-47ea-a7e5-03e9420ddea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:37.589699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46056af0-d29b-47ea-a7e5-03e9420ddea2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:37.590218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46056af0-d29b-47ea-a7e5-03e9420ddea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:37.590730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46056af0-d29b-47ea-a7e5-03e9420ddea2 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:37.592211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46056af0-d29b-47ea-a7e5-03e9420ddea2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:37.593367 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46056af0-d29b-47ea-a7e5-03e9420ddea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:37.595464 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46056af0-d29b-47ea-a7e5-03e9420ddea2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:37.619560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 614/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jul 21 08:59:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:37.620898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52609fcf-1c3d-4fff-b6ef-4d3efac7d240 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:37.749106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:37.770448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52609fcf-1c3d-4fff-b6ef-4d3efac7d240 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:37.808218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f36239-c9da-4ff2-90b7-3cc375d24fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:37.808895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f36239-c9da-4ff2-90b7-3cc375d24fcd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:37.809174 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f36239-c9da-4ff2-90b7-3cc375d24fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:37.809447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f36239-c9da-4ff2-90b7-3cc375d24fcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:37.823254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f36239-c9da-4ff2-90b7-3cc375d24fcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:37.825227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f36239-c9da-4ff2-90b7-3cc375d24fcd None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:37.844925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-a0f36239-c9da-4ff2-90b7-3cc375d24fcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:37.844925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:37.844925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:37.844925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:37.844925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:37.844925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:37.844925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:37.844925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:37.844925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:37.844925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:37.844925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:37.844925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:37.844925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:59:37.844925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:59:37.844925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:37.846805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:37.847696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:59:37.847696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:59:37.847696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:37.847696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:37.847696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 623/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:59:37.852798 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46056af0-d29b-47ea-a7e5-03e9420ddea2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:38.008649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46056af0-d29b-47ea-a7e5-03e9420ddea2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:38.019174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:38.047901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f12425-4f58-45bf-b0a7-9a3923f0855e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:38.048775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f12425-4f58-45bf-b0a7-9a3923f0855e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:38.049094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f12425-4f58-45bf-b0a7-9a3923f0855e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:38.049416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f12425-4f58-45bf-b0a7-9a3923f0855e None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:38.050157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f12425-4f58-45bf-b0a7-9a3923f0855e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:38.050470 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f12425-4f58-45bf-b0a7-9a3923f0855e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:38.051884 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f12425-4f58-45bf-b0a7-9a3923f0855e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:38.074869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 615/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:38.076024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f36239-c9da-4ff2-90b7-3cc375d24fcd None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:38.206477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f36239-c9da-4ff2-90b7-3cc375d24fcd None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:38.251032 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4092e1-bcd5-40fd-be24-554ed4aecef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:38.252164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4092e1-bcd5-40fd-be24-554ed4aecef9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:38.252451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4092e1-bcd5-40fd-be24-554ed4aecef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:38.252728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4092e1-bcd5-40fd-be24-554ed4aecef9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:38.256987 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:38.267245 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4092e1-bcd5-40fd-be24-554ed4aecef9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:38.270167 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4092e1-bcd5-40fd-be24-554ed4aecef9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-8a4092e1-bcd5-40fd-be24-554ed4aecef9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:59:38.290322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:38.291583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:38.292643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:38.292643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:38.292643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:59:38.292643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:59:38.292643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:59:38.292643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:59:38.292643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:59:38.292643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:59:38.292643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:59:38.292643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:38.292643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 624/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:38.298373 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f12425-4f58-45bf-b0a7-9a3923f0855e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:38.423936 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f12425-4f58-45bf-b0a7-9a3923f0855e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:38.463342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03c073-a13e-4942-9026-bbffeaad4a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:38.464084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03c073-a13e-4942-9026-bbffeaad4a9d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:38.464496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03c073-a13e-4942-9026-bbffeaad4a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:38.464800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03c073-a13e-4942-9026-bbffeaad4a9d None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:38.465652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03c073-a13e-4942-9026-bbffeaad4a9d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:38.465973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03c073-a13e-4942-9026-bbffeaad4a9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:38.467672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03c073-a13e-4942-9026-bbffeaad4a9d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:38.491985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 616/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jul 21 08:59:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:38.495364 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4092e1-bcd5-40fd-be24-554ed4aecef9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:38.613491 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4092e1-bcd5-40fd-be24-554ed4aecef9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:38.671870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645507a-beac-489e-b2ae-cb6cc89fc5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:38.673631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645507a-beac-489e-b2ae-cb6cc89fc5cf None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:38.674147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645507a-beac-489e-b2ae-cb6cc89fc5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:38.674624 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645507a-beac-489e-b2ae-cb6cc89fc5cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:38.754354 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:38.762056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645507a-beac-489e-b2ae-cb6cc89fc5cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:38.763636 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645507a-beac-489e-b2ae-cb6cc89fc5cf None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:38.891763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 625/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:59:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:59:38.898121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03c073-a13e-4942-9026-bbffeaad4a9d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:38.940718 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:39.053164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03c073-a13e-4942-9026-bbffeaad4a9d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:39.107291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b90c2-a37e-45c9-b4d1-7e418db31c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:39.108159 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b90c2-a37e-45c9-b4d1-7e418db31c1d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:39.108470 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b90c2-a37e-45c9-b4d1-7e418db31c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:39.108740 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b90c2-a37e-45c9-b4d1-7e418db31c1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:39.124008 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b90c2-a37e-45c9-b4d1-7e418db31c1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:39.125913 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b90c2-a37e-45c9-b4d1-7e418db31c1d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:39.149973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-496b90c2-a37e-45c9-b4d1-7e418db31c1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:39.149973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:39.149973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:39.149973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:39.149973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:39.149973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:39.149973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:39.149973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:39.149973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:39.149973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:39.149973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:39.149973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:39.149973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:59:39.149973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:59:39.149973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:39.151327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:39.152763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:59:39.152763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:59:39.152763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:39.152763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:39.152763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 617/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:39.161227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645507a-beac-489e-b2ae-cb6cc89fc5cf None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:39.284613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:39.291728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645507a-beac-489e-b2ae-cb6cc89fc5cf None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:39.341242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82daa04e-45b3-4ec3-ba27-2b0518f7aadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:39.342402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82daa04e-45b3-4ec3-ba27-2b0518f7aadc None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:39.342867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82daa04e-45b3-4ec3-ba27-2b0518f7aadc None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:39.343485 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82daa04e-45b3-4ec3-ba27-2b0518f7aadc None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:39.345363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82daa04e-45b3-4ec3-ba27-2b0518f7aadc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:39.345909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82daa04e-45b3-4ec3-ba27-2b0518f7aadc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:39.353685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82daa04e-45b3-4ec3-ba27-2b0518f7aadc None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:39.370704 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 626/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:59:39.380223 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b90c2-a37e-45c9-b4d1-7e418db31c1d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:39.495944 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:39.521717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b90c2-a37e-45c9-b4d1-7e418db31c1d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:39.564122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7910fa6-a40b-4b1b-b82c-1fcef7f1dc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:39.564696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7910fa6-a40b-4b1b-b82c-1fcef7f1dc79 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:39.564971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7910fa6-a40b-4b1b-b82c-1fcef7f1dc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:39.565241 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7910fa6-a40b-4b1b-b82c-1fcef7f1dc79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:39.573597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7910fa6-a40b-4b1b-b82c-1fcef7f1dc79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:39.575240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7910fa6-a40b-4b1b-b82c-1fcef7f1dc79 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:39.592667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-d7910fa6-a40b-4b1b-b82c-1fcef7f1dc79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:59:39.592667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:39.592667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:39.592667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:39.592667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:39.592667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:39.592667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:39.592667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:39.592667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:39.592667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:39.592667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:39.592667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:39.592667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:59:39.592667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:59:39.592667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:39.593766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:39.595807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:59:39.595807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:59:39.595807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:59:39.595807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:59:39.595807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:59:39.595807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:59:39.595807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:59:39.595807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:39.595807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 618/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:59:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:39.601189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82daa04e-45b3-4ec3-ba27-2b0518f7aadc None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:39.727842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82daa04e-45b3-4ec3-ba27-2b0518f7aadc None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:39.785873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65e51a-62e7-47a0-8ee8-1c7a16d7fe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:39.785873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65e51a-62e7-47a0-8ee8-1c7a16d7fe52 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:39.785873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65e51a-62e7-47a0-8ee8-1c7a16d7fe52 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:39.785873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65e51a-62e7-47a0-8ee8-1c7a16d7fe52 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:39.786584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65e51a-62e7-47a0-8ee8-1c7a16d7fe52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:39.786890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65e51a-62e7-47a0-8ee8-1c7a16d7fe52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:39.788411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65e51a-62e7-47a0-8ee8-1c7a16d7fe52 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:39.803462 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 627/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:59:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:39.809372 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7910fa6-a40b-4b1b-b82c-1fcef7f1dc79 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:39.955599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7910fa6-a40b-4b1b-b82c-1fcef7f1dc79 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:40.006006 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f40946-99d6-4b63-a867-05885b16a0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:40.006678 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f40946-99d6-4b63-a867-05885b16a0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:40.006970 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f40946-99d6-4b63-a867-05885b16a0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:40.007244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f40946-99d6-4b63-a867-05885b16a0c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:40.008140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f40946-99d6-4b63-a867-05885b16a0c7 None admin] RBAC: Adding query filter params (role.id=8511cf3e5d2c4acdae6d0ffda5e72c63, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=72f9ef6de66e456e800237e9f954e98d) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:40.008453 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f40946-99d6-4b63-a867-05885b16a0c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:40.009937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f40946-99d6-4b63-a867-05885b16a0c7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:40.018919 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 619/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:59:39 2022] GET /identity/v3/role_assignments?user.id=72f9ef6de66e456e800237e9f954e98d&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=8511cf3e5d2c4acdae6d0ffda5e72c63 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:42.463147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1558a0-9255-4dd7-8ad6-015c0e454c4a None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:42.463773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1558a0-9255-4dd7-8ad6-015c0e454c4a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:42.464044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1558a0-9255-4dd7-8ad6-015c0e454c4a None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:42.467352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 628/1247] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:59:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:59:42.473260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ed0e1-6889-410f-94a2-fb99414d6c31 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:42.473777 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ed0e1-6889-410f-94a2-fb99414d6c31 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:42.474242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ed0e1-6889-410f-94a2-fb99414d6c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:42.484036 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:42.505964 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b63ed0e1-6889-410f-94a2-fb99414d6c31 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:59:42.507373 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b63ed0e1-6889-410f-94a2-fb99414d6c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:59:42.588613 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 620/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:59:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:59:42.618078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1558a0-9255-4dd7-8ad6-015c0e454c4a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:42.638859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:42.743556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1558a0-9255-4dd7-8ad6-015c0e454c4a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:42.785702 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5e48b-066b-4af9-ae3d-accd6dc678bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:42.786506 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5e48b-066b-4af9-ae3d-accd6dc678bd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:42.786805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5e48b-066b-4af9-ae3d-accd6dc678bd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:42.787166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5e48b-066b-4af9-ae3d-accd6dc678bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:42.800678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5e48b-066b-4af9-ae3d-accd6dc678bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-9fd5e48b-066b-4af9-ae3d-accd6dc678bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:59:42.809682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:42.810795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:42.811868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:42.811868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:42.811868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:42.811868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:42.811868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:59:42.811868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:59:42.811868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:59:42.811868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:59:42.811868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:59:42.811868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:59:42.811868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:59:42.811868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:42.811868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 629/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:59:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:42.817951 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ed0e1-6889-410f-94a2-fb99414d6c31 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:42.947545 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ed0e1-6889-410f-94a2-fb99414d6c31 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:42.986325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:42.992216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828badd4-1762-41ad-9539-0aa35186cd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:42.993018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828badd4-1762-41ad-9539-0aa35186cd1b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:42.995068 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828badd4-1762-41ad-9539-0aa35186cd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:42.995395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828badd4-1762-41ad-9539-0aa35186cd1b None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:42.996345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828badd4-1762-41ad-9539-0aa35186cd1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:42.996819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828badd4-1762-41ad-9539-0aa35186cd1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:42.998350 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828badd4-1762-41ad-9539-0aa35186cd1b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:43.008445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 621/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:59:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:43.014671 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5e48b-066b-4af9-ae3d-accd6dc678bd None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:43.112469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:43.140253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5e48b-066b-4af9-ae3d-accd6dc678bd None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:43.181754 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40493570-38af-4fa3-b3a8-39eef1610bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:43.182855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40493570-38af-4fa3-b3a8-39eef1610bde None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:43.183142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40493570-38af-4fa3-b3a8-39eef1610bde None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:43.183411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40493570-38af-4fa3-b3a8-39eef1610bde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:43.196393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40493570-38af-4fa3-b3a8-39eef1610bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:43.197899 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40493570-38af-4fa3-b3a8-39eef1610bde None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:43.216573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-40493570-38af-4fa3-b3a8-39eef1610bde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:43.216573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:43.216573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:43.216573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:43.216573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:43.216573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:43.216573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:43.216573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:43.216573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:43.216573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:43.216573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:43.216573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:43.216573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:59:43.216573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:59:43.216573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:43.217641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:43.218537 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:59:43.218537 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:59:43.218537 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:43.218537 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:43.218537 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 630/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:59:43.224732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828badd4-1762-41ad-9539-0aa35186cd1b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:43.356744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828badd4-1762-41ad-9539-0aa35186cd1b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:43.400745 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a18df-f4be-436e-af62-79679e8689ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:43.401845 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a18df-f4be-436e-af62-79679e8689ec None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:43.402294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a18df-f4be-436e-af62-79679e8689ec None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:43.402763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a18df-f4be-436e-af62-79679e8689ec None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:43.403760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a18df-f4be-436e-af62-79679e8689ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:43.404193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a18df-f4be-436e-af62-79679e8689ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:43.405765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a18df-f4be-436e-af62-79679e8689ec None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:43.420680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 622/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:43.427467 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40493570-38af-4fa3-b3a8-39eef1610bde None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:43.556555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40493570-38af-4fa3-b3a8-39eef1610bde None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:43.590606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:43.600531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7bfd3-42a2-430d-9b2b-fbd18a4ed610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:43.601367 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7bfd3-42a2-430d-9b2b-fbd18a4ed610 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:43.601654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7bfd3-42a2-430d-9b2b-fbd18a4ed610 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:43.601930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7bfd3-42a2-430d-9b2b-fbd18a4ed610 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:43.611395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7bfd3-42a2-430d-9b2b-fbd18a4ed610 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:43.612870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e7bfd3-42a2-430d-9b2b-fbd18a4ed610 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-99e7bfd3-42a2-430d-9b2b-fbd18a4ed610 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:59:43.629343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:43.630559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:43.633325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:43.633325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:43.633325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:59:43.633325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:59:43.633325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:59:43.633325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:59:43.633325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:59:43.633325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:59:43.633325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:59:43.633325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:43.633325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 631/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:59:43.636918 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a18df-f4be-436e-af62-79679e8689ec None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:43.710231 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:43.778961 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a18df-f4be-436e-af62-79679e8689ec None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:43.831049 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c3d50-666b-475d-b7b7-fa7db3ba229d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:43.831750 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c3d50-666b-475d-b7b7-fa7db3ba229d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:43.832087 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c3d50-666b-475d-b7b7-fa7db3ba229d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:43.832355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c3d50-666b-475d-b7b7-fa7db3ba229d None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:43.833445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c3d50-666b-475d-b7b7-fa7db3ba229d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:43.833784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c3d50-666b-475d-b7b7-fa7db3ba229d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:43.835306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c3d50-666b-475d-b7b7-fa7db3ba229d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:43.857456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 623/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jul 21 08:59:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:43.864750 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7bfd3-42a2-430d-9b2b-fbd18a4ed610 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:44.024571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7bfd3-42a2-430d-9b2b-fbd18a4ed610 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:44.069869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad8c1b-f8cb-489e-8840-6db094ba6e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:44.070706 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad8c1b-f8cb-489e-8840-6db094ba6e9c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:44.070992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad8c1b-f8cb-489e-8840-6db094ba6e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:44.071290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad8c1b-f8cb-489e-8840-6db094ba6e9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:44.149804 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:44.168160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad8c1b-f8cb-489e-8840-6db094ba6e9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:44.170201 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad8c1b-f8cb-489e-8840-6db094ba6e9c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:44.303995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 632/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:59:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:59:44.305246 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c3d50-666b-475d-b7b7-fa7db3ba229d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:44.415855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c3d50-666b-475d-b7b7-fa7db3ba229d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:44.434063 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:44.464149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52da05-d305-4629-a07d-2a3fb6a10fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:44.465155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52da05-d305-4629-a07d-2a3fb6a10fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:44.465618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52da05-d305-4629-a07d-2a3fb6a10fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:44.465948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52da05-d305-4629-a07d-2a3fb6a10fe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:44.484726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52da05-d305-4629-a07d-2a3fb6a10fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:44.486930 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52da05-d305-4629-a07d-2a3fb6a10fe6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:44.510737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-7c52da05-d305-4629-a07d-2a3fb6a10fe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:44.510737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:44.510737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:44.510737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:44.510737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:44.510737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:44.510737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:44.510737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:44.510737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:44.510737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:44.510737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:44.510737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:44.510737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:59:44.510737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:59:44.510737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:44.511879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:44.512920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:59:44.512920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:59:44.512920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:44.512920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:44.512920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 624/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:44.520247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad8c1b-f8cb-489e-8840-6db094ba6e9c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:44.621792 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:44.636976 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad8c1b-f8cb-489e-8840-6db094ba6e9c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:44.674456 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ec77f-9786-4559-ba01-22b137ac4a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:44.675103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ec77f-9786-4559-ba01-22b137ac4a69 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:44.675412 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ec77f-9786-4559-ba01-22b137ac4a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:44.675685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ec77f-9786-4559-ba01-22b137ac4a69 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:44.676504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ec77f-9786-4559-ba01-22b137ac4a69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:44.676809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ec77f-9786-4559-ba01-22b137ac4a69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:44.678449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ec77f-9786-4559-ba01-22b137ac4a69 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:44.692547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 633/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:44.698511 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52da05-d305-4629-a07d-2a3fb6a10fe6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:44.825141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52da05-d305-4629-a07d-2a3fb6a10fe6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:44.865812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d50c8d-cff2-4549-93bf-f73d185cef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:44.866788 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d50c8d-cff2-4549-93bf-f73d185cef50 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:44.867117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d50c8d-cff2-4549-93bf-f73d185cef50 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:44.867546 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d50c8d-cff2-4549-93bf-f73d185cef50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:44.878107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d50c8d-cff2-4549-93bf-f73d185cef50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:44.880335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d50c8d-cff2-4549-93bf-f73d185cef50 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:44.902358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-23d50c8d-cff2-4549-93bf-f73d185cef50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:59:44.902358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:44.902358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:44.902358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:44.902358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:44.902358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:44.902358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:44.902358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:44.902358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:44.902358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:44.902358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:44.902358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:44.902358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:59:44.902358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:59:44.902358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:44.903654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:44.904999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:59:44.904999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:59:44.904999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:59:44.904999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:59:44.904999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:59:44.904999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:59:44.904999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:59:44.904999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:44.904999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 625/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:59:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:44.910408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ec77f-9786-4559-ba01-22b137ac4a69 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:45.034357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ec77f-9786-4559-ba01-22b137ac4a69 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:45.087508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6e80e-6283-4d39-bf3a-6d5930e6c392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:45.088207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6e80e-6283-4d39-bf3a-6d5930e6c392 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:45.088494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6e80e-6283-4d39-bf3a-6d5930e6c392 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:45.088769 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6e80e-6283-4d39-bf3a-6d5930e6c392 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:45.089559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6e80e-6283-4d39-bf3a-6d5930e6c392 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:45.089874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6e80e-6283-4d39-bf3a-6d5930e6c392 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:45.091312 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6e80e-6283-4d39-bf3a-6d5930e6c392 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:45.104890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 634/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:59:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:59:45.111026 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d50c8d-cff2-4549-93bf-f73d185cef50 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:45.139522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:45.222562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d50c8d-cff2-4549-93bf-f73d185cef50 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:45.264222 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aac1f4-aa81-4c0c-9565-b34209cbc7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:45.264887 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aac1f4-aa81-4c0c-9565-b34209cbc7ee None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:45.265235 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aac1f4-aa81-4c0c-9565-b34209cbc7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:45.265575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aac1f4-aa81-4c0c-9565-b34209cbc7ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:45.267139 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aac1f4-aa81-4c0c-9565-b34209cbc7ee None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=72f9ef6de66e456e800237e9f954e98d) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:45.267475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aac1f4-aa81-4c0c-9565-b34209cbc7ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:45.269165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aac1f4-aa81-4c0c-9565-b34209cbc7ee None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:45.275592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 626/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:59:45 2022] GET /identity/v3/role_assignments?user.id=72f9ef6de66e456e800237e9f954e98d&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=000e942517f2469ba36d2e65978bc314 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:47.474962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf8740-86f3-4313-9e7c-37bcaf766301 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:47.475524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf8740-86f3-4313-9e7c-37bcaf766301 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:47.475524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf8740-86f3-4313-9e7c-37bcaf766301 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:47.476986 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 635/1261] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:59:47.482920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbbc3d-2699-47df-a317-5c8006159a4a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:47.483472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbbc3d-2699-47df-a317-5c8006159a4a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:47.483472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbbc3d-2699-47df-a317-5c8006159a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:47.502418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5ecbbc3d-2699-47df-a317-5c8006159a4a None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:59:47.503676 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5ecbbc3d-2699-47df-a317-5c8006159a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:59:47.595794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 627/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:59:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:59:47.618780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf8740-86f3-4313-9e7c-37bcaf766301 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:47.630766 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:47.725781 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf8740-86f3-4313-9e7c-37bcaf766301 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:47.760822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb733ea2-8fd8-41f8-a11c-8da35c93d2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:47.761602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb733ea2-8fd8-41f8-a11c-8da35c93d2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:47.761879 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb733ea2-8fd8-41f8-a11c-8da35c93d2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:47.762147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb733ea2-8fd8-41f8-a11c-8da35c93d2f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:47.772750 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb733ea2-8fd8-41f8-a11c-8da35c93d2f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-bb733ea2-8fd8-41f8-a11c-8da35c93d2f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:59:47.780281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:47.781344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:47.782651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:47.782651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:47.782651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:47.782651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:47.782651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:59:47.782651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:59:47.782651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:59:47.782651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:59:47.782651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:59:47.782651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:59:47.782651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:59:47.782651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:47.782651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 636/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:59:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:47.787802 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbbc3d-2699-47df-a317-5c8006159a4a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:47.806972 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:47.897539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbbc3d-2699-47df-a317-5c8006159a4a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:47.938259 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd439d1e-d249-4231-b168-37f632c7593f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:47.939024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd439d1e-d249-4231-b168-37f632c7593f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:47.939303 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd439d1e-d249-4231-b168-37f632c7593f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:47.939579 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd439d1e-d249-4231-b168-37f632c7593f None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:47.940355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd439d1e-d249-4231-b168-37f632c7593f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:47.940660 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd439d1e-d249-4231-b168-37f632c7593f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:47.942037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd439d1e-d249-4231-b168-37f632c7593f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:47.950897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 628/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:59:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:47.956858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb733ea2-8fd8-41f8-a11c-8da35c93d2f5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:48.030933 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:48.066782 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb733ea2-8fd8-41f8-a11c-8da35c93d2f5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:48.102351 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e749243-0165-4bce-b4c3-14c5a14e63f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:48.103141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e749243-0165-4bce-b4c3-14c5a14e63f2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:48.103424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e749243-0165-4bce-b4c3-14c5a14e63f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:48.103697 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e749243-0165-4bce-b4c3-14c5a14e63f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:48.114972 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e749243-0165-4bce-b4c3-14c5a14e63f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:48.116458 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e749243-0165-4bce-b4c3-14c5a14e63f2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:48.134074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-9e749243-0165-4bce-b4c3-14c5a14e63f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:48.134074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:48.134074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:48.134074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:48.134074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:48.134074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:48.134074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:48.134074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:48.134074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:48.134074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:48.134074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:48.134074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:48.134074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:59:48.134074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:59:48.134074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:48.135177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:48.136156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:59:48.136156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:59:48.136156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:48.136156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:48.136156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 637/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:48.142070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd439d1e-d249-4231-b168-37f632c7593f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:48.225321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:48.246256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd439d1e-d249-4231-b168-37f632c7593f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:48.283542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47596cc-229c-4e9e-8a7f-c9cbd379a6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:48.284139 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47596cc-229c-4e9e-8a7f-c9cbd379a6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:48.284425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47596cc-229c-4e9e-8a7f-c9cbd379a6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:48.284691 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47596cc-229c-4e9e-8a7f-c9cbd379a6d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:48.285411 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47596cc-229c-4e9e-8a7f-c9cbd379a6d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:48.285716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47596cc-229c-4e9e-8a7f-c9cbd379a6d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:48.287095 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47596cc-229c-4e9e-8a7f-c9cbd379a6d7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:48.299349 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 629/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:59:48.305475 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e749243-0165-4bce-b4c3-14c5a14e63f2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:48.414789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e749243-0165-4bce-b4c3-14c5a14e63f2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:48.435549 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:48.450840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4a548-ca08-41a4-b240-5569d44b4fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:48.451635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4a548-ca08-41a4-b240-5569d44b4fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:48.451931 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4a548-ca08-41a4-b240-5569d44b4fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:48.452210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4a548-ca08-41a4-b240-5569d44b4fa6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:48.461763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4a548-ca08-41a4-b240-5569d44b4fa6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:48.463239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4a548-ca08-41a4-b240-5569d44b4fa6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-30b4a548-ca08-41a4-b240-5569d44b4fa6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:59:48.479863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:48.481477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:48.482614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:48.482614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:48.482614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:59:48.482614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:59:48.482614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:59:48.482614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:59:48.482614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:59:48.482614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:59:48.482614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:59:48.482614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:48.482614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 638/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:48.489582 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47596cc-229c-4e9e-8a7f-c9cbd379a6d7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:48.594727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47596cc-229c-4e9e-8a7f-c9cbd379a6d7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:48.632790 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c47bb-b650-44cb-8269-676dbcbd5832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:48.633623 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c47bb-b650-44cb-8269-676dbcbd5832 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:48.634032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c47bb-b650-44cb-8269-676dbcbd5832 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:48.634455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c47bb-b650-44cb-8269-676dbcbd5832 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:48.635444 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c47bb-b650-44cb-8269-676dbcbd5832 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:48.635870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c47bb-b650-44cb-8269-676dbcbd5832 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:48.637500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c47bb-b650-44cb-8269-676dbcbd5832 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:48.646984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:48.656069 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 630/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jul 21 08:59:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:48.662772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4a548-ca08-41a4-b240-5569d44b4fa6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:48.767059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4a548-ca08-41a4-b240-5569d44b4fa6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:48.800875 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4128ef2-1f3b-47b7-9eb3-adebb5ecc616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:48.801665 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4128ef2-1f3b-47b7-9eb3-adebb5ecc616 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:48.801939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4128ef2-1f3b-47b7-9eb3-adebb5ecc616 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:48.802469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4128ef2-1f3b-47b7-9eb3-adebb5ecc616 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:48.813728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4128ef2-1f3b-47b7-9eb3-adebb5ecc616 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:48.815188 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4128ef2-1f3b-47b7-9eb3-adebb5ecc616 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:48.833235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-b4128ef2-1f3b-47b7-9eb3-adebb5ecc616 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:48.833235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:48.833235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:48.833235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:48.833235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:48.833235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:48.833235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:48.833235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:48.833235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:48.833235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:48.833235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:48.833235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:48.833235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:59:48.833235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:59:48.833235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:48.834334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:48.835394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:59:48.835394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:59:48.835394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:48.835394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:48.835394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 639/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:48.840433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c47bb-b650-44cb-8269-676dbcbd5832 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:48.948689 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c47bb-b650-44cb-8269-676dbcbd5832 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:48.983490 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6340f-af17-4f41-9b8f-fb3014b58459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:48.984182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6340f-af17-4f41-9b8f-fb3014b58459 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:48.984508 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6340f-af17-4f41-9b8f-fb3014b58459 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:48.984787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c6340f-af17-4f41-9b8f-fb3014b58459 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:48.985508 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6340f-af17-4f41-9b8f-fb3014b58459 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:48.985815 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6340f-af17-4f41-9b8f-fb3014b58459 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:48.987206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c6340f-af17-4f41-9b8f-fb3014b58459 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:48.998568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 631/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:49.004461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4128ef2-1f3b-47b7-9eb3-adebb5ecc616 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:49.036772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:49.107221 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4128ef2-1f3b-47b7-9eb3-adebb5ecc616 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:49.142425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e8717-6d17-449c-a276-5db23c2cf894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:49.143223 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e8717-6d17-449c-a276-5db23c2cf894 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:49.143502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e8717-6d17-449c-a276-5db23c2cf894 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:49.143773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e8717-6d17-449c-a276-5db23c2cf894 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:49.151009 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e8717-6d17-449c-a276-5db23c2cf894 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:49.152485 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04e8717-6d17-449c-a276-5db23c2cf894 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:49.167089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-a04e8717-6d17-449c-a276-5db23c2cf894 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:59:49.167089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:49.167089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:49.167089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:49.167089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:49.167089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:49.167089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:49.167089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:49.167089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:49.167089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:49.167089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:49.167089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:49.167089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:59:49.167089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:59:49.167089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:49.168215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:49.169538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:59:49.169538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:59:49.169538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:59:49.169538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:59:49.169538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:59:49.169538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:59:49.169538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:59:49.169538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:49.169538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 640/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:59:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:49.174930 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6340f-af17-4f41-9b8f-fb3014b58459 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:49.236815 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:49.279589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c6340f-af17-4f41-9b8f-fb3014b58459 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:49.315562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56320f2-7fdb-408e-8e22-e7d70e85bc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:49.316249 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56320f2-7fdb-408e-8e22-e7d70e85bc41 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:49.316532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56320f2-7fdb-408e-8e22-e7d70e85bc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:49.316808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56320f2-7fdb-408e-8e22-e7d70e85bc41 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:49.317597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56320f2-7fdb-408e-8e22-e7d70e85bc41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:49.317908 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56320f2-7fdb-408e-8e22-e7d70e85bc41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:49.319372 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56320f2-7fdb-408e-8e22-e7d70e85bc41 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:49.332128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 632/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:59:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:49.338310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e8717-6d17-449c-a276-5db23c2cf894 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:49.431374 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:49.441057 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04e8717-6d17-449c-a276-5db23c2cf894 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:49.476352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b3e97-32b4-43d4-ac48-110860f55bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:49.477247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b3e97-32b4-43d4-ac48-110860f55bf6 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:49.477526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b3e97-32b4-43d4-ac48-110860f55bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:49.477802 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b3e97-32b4-43d4-ac48-110860f55bf6 None admin] PATH_INFO: `/v3/projects/7fe0f489cbce43859f050c3270e00709/users/72f9ef6de66e456e800237e9f954e98d/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:49.479752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b3e97-32b4-43d4-ac48-110860f55bf6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fe0f489cbce43859f050c3270e00709, user_id=72f9ef6de66e456e800237e9f954e98d, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:49.481228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b3e97-32b4-43d4-ac48-110860f55bf6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:49.494974 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 641/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jul 21 08:59:49 2022] PUT /identity/v3/projects/7fe0f489cbce43859f050c3270e00709/users/72f9ef6de66e456e800237e9f954e98d/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:59:51.550126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483fa77-8da3-4332-be7f-7240fd3d74af None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:51.550662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483fa77-8da3-4332-be7f-7240fd3d74af None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:51.550662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483fa77-8da3-4332-be7f-7240fd3d74af None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:51.552084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 633/1274] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:59:51.557570 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f0b01-a80d-4e98-9078-3fe8f17974db None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:51.558048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f0b01-a80d-4e98-9078-3fe8f17974db None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:51.558141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f0b01-a80d-4e98-9078-3fe8f17974db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:51.574756 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-871f0b01-a80d-4e98-9078-3fe8f17974db None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:59:51.575977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-871f0b01-a80d-4e98-9078-3fe8f17974db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:59:51.645252 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 642/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:59:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:59:51.671666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483fa77-8da3-4332-be7f-7240fd3d74af None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:51.783287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483fa77-8da3-4332-be7f-7240fd3d74af None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:51.801955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:51.820391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c3c77-0f85-4594-9f6a-bc00de0ccfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:51.821172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c3c77-0f85-4594-9f6a-bc00de0ccfe0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:51.821455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c3c77-0f85-4594-9f6a-bc00de0ccfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:51.821730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c3c77-0f85-4594-9f6a-bc00de0ccfe0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:51.832552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c3c77-0f85-4594-9f6a-bc00de0ccfe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-0b2c3c77-0f85-4594-9f6a-bc00de0ccfe0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:59:51.840218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:51.841294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:51.842287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:51.842287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:51.842287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:51.842287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:51.842287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:59:51.842287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:59:51.842287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:59:51.842287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:59:51.842287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:59:51.842287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:59:51.842287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:59:51.842287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:51.842287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 634/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jul 21 08:59:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:51.848175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f0b01-a80d-4e98-9078-3fe8f17974db None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:51.952111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:51.978240 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f0b01-a80d-4e98-9078-3fe8f17974db None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:52.017765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d500795-9d43-4130-856d-bb4e21b30d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:52.018456 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d500795-9d43-4130-856d-bb4e21b30d73 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:52.018740 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d500795-9d43-4130-856d-bb4e21b30d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:52.019017 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d500795-9d43-4130-856d-bb4e21b30d73 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:52.019785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d500795-9d43-4130-856d-bb4e21b30d73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:52.020133 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d500795-9d43-4130-856d-bb4e21b30d73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:52.021551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d500795-9d43-4130-856d-bb4e21b30d73 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:52.030792 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 643/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jul 21 08:59:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:52.037353 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c3c77-0f85-4594-9f6a-bc00de0ccfe0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:52.145259 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c3c77-0f85-4594-9f6a-bc00de0ccfe0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:52.184724 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93b8a8-f1ca-41ae-a450-eb62b3dff34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:52.185509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93b8a8-f1ca-41ae-a450-eb62b3dff34d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:52.185791 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93b8a8-f1ca-41ae-a450-eb62b3dff34d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:52.186064 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93b8a8-f1ca-41ae-a450-eb62b3dff34d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:52.198200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93b8a8-f1ca-41ae-a450-eb62b3dff34d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:52.199691 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93b8a8-f1ca-41ae-a450-eb62b3dff34d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:52.217242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-6f93b8a8-f1ca-41ae-a450-eb62b3dff34d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:52.217242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:52.217242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:52.217242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:52.217242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:52.217242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:52.217242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:52.217242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:52.217242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:52.217242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:52.217242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:52.217242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:52.217242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:59:52.217242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:59:52.217242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:52.218331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:52.219462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:59:52.219462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:59:52.219462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:52.219462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:52.219462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 635/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:52.225370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d500795-9d43-4130-856d-bb4e21b30d73 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:52.325393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d500795-9d43-4130-856d-bb4e21b30d73 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:52.359141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839849e-8c12-41be-ae44-fe6031f651a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:52.360097 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839849e-8c12-41be-ae44-fe6031f651a4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:52.360380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839849e-8c12-41be-ae44-fe6031f651a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:52.360652 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839849e-8c12-41be-ae44-fe6031f651a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:52.361380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839849e-8c12-41be-ae44-fe6031f651a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:52.361683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839849e-8c12-41be-ae44-fe6031f651a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:52.363058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839849e-8c12-41be-ae44-fe6031f651a4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:52.379117 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 644/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:52.384897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93b8a8-f1ca-41ae-a450-eb62b3dff34d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:52.418850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:52.493683 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93b8a8-f1ca-41ae-a450-eb62b3dff34d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:52.528004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c297b8-affc-4c38-82b2-33280fcca47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:52.528809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c297b8-affc-4c38-82b2-33280fcca47b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:52.529094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c297b8-affc-4c38-82b2-33280fcca47b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:52.529369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c297b8-affc-4c38-82b2-33280fcca47b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:52.537957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c297b8-affc-4c38-82b2-33280fcca47b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:52.539433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c297b8-affc-4c38-82b2-33280fcca47b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-c3c297b8-affc-4c38-82b2-33280fcca47b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:59:52.554369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:52.555979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:52.556967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:52.556967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:52.556967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:59:52.556967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:59:52.556967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:59:52.556967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:59:52.556967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:59:52.556967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:59:52.556967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:59:52.556967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:52.556967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 636/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:52.561501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839849e-8c12-41be-ae44-fe6031f651a4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:52.566295 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:52.665971 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839849e-8c12-41be-ae44-fe6031f651a4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:52.700302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba902d-7204-4834-95e3-f6e73eacc8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:52.700894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba902d-7204-4834-95e3-f6e73eacc8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:52.701180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba902d-7204-4834-95e3-f6e73eacc8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:52.701452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba902d-7204-4834-95e3-f6e73eacc8a6 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:52.702309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba902d-7204-4834-95e3-f6e73eacc8a6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:52.702615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba902d-7204-4834-95e3-f6e73eacc8a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:52.704651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba902d-7204-4834-95e3-f6e73eacc8a6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:52.728006 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 645/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jul 21 08:59:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:52.733973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c297b8-affc-4c38-82b2-33280fcca47b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:52.828667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:52.839816 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c297b8-affc-4c38-82b2-33280fcca47b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:52.876954 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751b60e-bed8-4624-97da-cce3c337594d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:52.877623 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751b60e-bed8-4624-97da-cce3c337594d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:52.877901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751b60e-bed8-4624-97da-cce3c337594d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:52.878366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751b60e-bed8-4624-97da-cce3c337594d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:52.960709 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751b60e-bed8-4624-97da-cce3c337594d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:52.962087 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751b60e-bed8-4624-97da-cce3c337594d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:53.057119 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:53.069089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 637/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jul 21 08:59:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:59:53.075186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba902d-7204-4834-95e3-f6e73eacc8a6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:53.145132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:53.186386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba902d-7204-4834-95e3-f6e73eacc8a6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:53.221525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b9b60-61f4-45f1-a767-587163f84cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:53.222235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b9b60-61f4-45f1-a767-587163f84cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:53.222520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b9b60-61f4-45f1-a767-587163f84cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:53.222795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b9b60-61f4-45f1-a767-587163f84cc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:53.233387 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b9b60-61f4-45f1-a767-587163f84cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:53.236032 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b9b60-61f4-45f1-a767-587163f84cc4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:53.254421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-a15b9b60-61f4-45f1-a767-587163f84cc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:53.254421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:53.254421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:53.254421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:53.254421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:53.254421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:53.254421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:53.254421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:53.254421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:53.254421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:53.254421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:53.254421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:53.254421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:59:53.254421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:59:53.254421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:53.255547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:53.256797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:59:53.256797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:59:53.256797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:59:53.256797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:53.256797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 646/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 08:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:53.261577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751b60e-bed8-4624-97da-cce3c337594d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:53.363270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751b60e-bed8-4624-97da-cce3c337594d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:53.402744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da6943-23cd-4128-b692-22a110cb6486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:53.403561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da6943-23cd-4128-b692-22a110cb6486 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:53.403894 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da6943-23cd-4128-b692-22a110cb6486 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:53.404202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da6943-23cd-4128-b692-22a110cb6486 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:53.404933 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da6943-23cd-4128-b692-22a110cb6486 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:53.405246 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da6943-23cd-4128-b692-22a110cb6486 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:53.406866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da6943-23cd-4128-b692-22a110cb6486 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:53.418551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 638/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jul 21 08:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:53.424825 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b9b60-61f4-45f1-a767-587163f84cc4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:53.538055 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b9b60-61f4-45f1-a767-587163f84cc4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:53.553783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:53.574532 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b044388-10d7-468c-b16c-ecaff7a17e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:53.575326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b044388-10d7-468c-b16c-ecaff7a17e2a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:53.575609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b044388-10d7-468c-b16c-ecaff7a17e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:53.575889 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b044388-10d7-468c-b16c-ecaff7a17e2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:53.583495 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b044388-10d7-468c-b16c-ecaff7a17e2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:53.584982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b044388-10d7-468c-b16c-ecaff7a17e2a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:53.599982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-4b044388-10d7-468c-b16c-ecaff7a17e2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:59:53.599982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:53.599982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:53.599982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:53.599982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:53.599982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:53.599982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:53.599982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:53.599982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:53.599982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:53.599982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:53.599982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:53.599982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:59:53.599982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:59:53.599982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:53.601107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:53.602301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:59:53.602301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:59:53.602301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:59:53.602301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:59:53.602301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:59:53.602301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:59:53.602301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:59:53.602301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 08:59:53.602301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 647/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jul 21 08:59:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:53.607345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da6943-23cd-4128-b692-22a110cb6486 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:53.660188 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:53.716910 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da6943-23cd-4128-b692-22a110cb6486 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:53.759233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60870090-d91f-4d19-ae4c-18d670c9db12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:53.759892 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60870090-d91f-4d19-ae4c-18d670c9db12 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:53.760204 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60870090-d91f-4d19-ae4c-18d670c9db12 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:53.760480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60870090-d91f-4d19-ae4c-18d670c9db12 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:53.761269 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60870090-d91f-4d19-ae4c-18d670c9db12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:53.761580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60870090-d91f-4d19-ae4c-18d670c9db12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:53.763032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60870090-d91f-4d19-ae4c-18d670c9db12 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:53.776502 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 639/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jul 21 08:59:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:53.783342 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b044388-10d7-468c-b16c-ecaff7a17e2a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:53.894632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b044388-10d7-468c-b16c-ecaff7a17e2a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:53.936309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0614d53-c710-4603-8c42-a3914b4bc6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:53.936961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0614d53-c710-4603-8c42-a3914b4bc6ea None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:53.937245 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0614d53-c710-4603-8c42-a3914b4bc6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:53.938417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0614d53-c710-4603-8c42-a3914b4bc6ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:53.939254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0614d53-c710-4603-8c42-a3914b4bc6ea None admin] RBAC: Adding query filter params (role.id=000e942517f2469ba36d2e65978bc314, scope.project.id=7fe0f489cbce43859f050c3270e00709, user.id=72f9ef6de66e456e800237e9f954e98d) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:53.939567 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0614d53-c710-4603-8c42-a3914b4bc6ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:53.941049 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0614d53-c710-4603-8c42-a3914b4bc6ea None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:53.947125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 648/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jul 21 08:59:53 2022] GET /identity/v3/role_assignments?user.id=72f9ef6de66e456e800237e9f954e98d&scope.project.id=7fe0f489cbce43859f050c3270e00709&role.id=000e942517f2469ba36d2e65978bc314 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:56.054435 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122912e2-f5d5-4ab4-a0ea-a716e4389add None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:56.054946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122912e2-f5d5-4ab4-a0ea-a716e4389add None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:56.054946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122912e2-f5d5-4ab4-a0ea-a716e4389add None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:56.056301 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 640/1288] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:59:56.061776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e6788-da9e-480f-a5fa-39e08194fcaf None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:56.062232 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e6788-da9e-480f-a5fa-39e08194fcaf None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:56.062968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e6788-da9e-480f-a5fa-39e08194fcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:56.080458 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-144e6788-da9e-480f-a5fa-39e08194fcaf None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:59:56.081707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-144e6788-da9e-480f-a5fa-39e08194fcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:59:56.113131 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:56.151886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 649/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:59:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:59:56.174829 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122912e2-f5d5-4ab4-a0ea-a716e4389add None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:56.291478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122912e2-f5d5-4ab4-a0ea-a716e4389add None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:56.303836 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:56.329293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3f310-d1b7-482c-b448-7340d31cbdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:56.330006 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3f310-d1b7-482c-b448-7340d31cbdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:56.330289 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3f310-d1b7-482c-b448-7340d31cbdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:56.330562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3f310-d1b7-482c-b448-7340d31cbdf4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:56.331225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3f310-d1b7-482c-b448-7340d31cbdf4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:56.332663 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3f310-d1b7-482c-b448-7340d31cbdf4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-f7a3f310-d1b7-482c-b448-7340d31cbdf4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:59:56.345416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:59:56.352166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:59:56.353306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:59:56.353306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 08:59:56.353306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:59:56.353306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:59:56.353306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:59:56.353306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:59:56.353306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:59:56.353306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:59:56.353306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:59:56.353306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:59:56.353306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:59:56.353306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:59:56.353306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 08:59:56.353306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 08:59:56.353306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 641/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jul 21 08:59:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:59:56.354129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e6788-da9e-480f-a5fa-39e08194fcaf None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:56.459737 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e6788-da9e-480f-a5fa-39e08194fcaf None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:56.494478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51670b-d24b-4af1-85a6-9cf0c1afef98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:56.495147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51670b-d24b-4af1-85a6-9cf0c1afef98 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:56.495461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51670b-d24b-4af1-85a6-9cf0c1afef98 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:56.495745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51670b-d24b-4af1-85a6-9cf0c1afef98 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:56.496502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51670b-d24b-4af1-85a6-9cf0c1afef98 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:56.496809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51670b-d24b-4af1-85a6-9cf0c1afef98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:56.498200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51670b-d24b-4af1-85a6-9cf0c1afef98 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:56.503193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 650/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Jul 21 08:59:56 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:56.509029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3f310-d1b7-482c-b448-7340d31cbdf4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:56.609849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3f310-d1b7-482c-b448-7340d31cbdf4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:56.644838 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79ad09-a41d-4189-8f4b-2061308b2699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:56.645506 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79ad09-a41d-4189-8f4b-2061308b2699 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:56.645793 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79ad09-a41d-4189-8f4b-2061308b2699 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:56.646072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79ad09-a41d-4189-8f4b-2061308b2699 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:56.646790 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79ad09-a41d-4189-8f4b-2061308b2699 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:59:56.647132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79ad09-a41d-4189-8f4b-2061308b2699 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:56.648571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79ad09-a41d-4189-8f4b-2061308b2699 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:56.653494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 642/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Jul 21 08:59:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:59:58.713063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8b9d5-d7fe-4103-8924-8c603cd6568f None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:58.713636 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8b9d5-d7fe-4103-8924-8c603cd6568f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:58.713636 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8b9d5-d7fe-4103-8924-8c603cd6568f None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:58.715018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 651/1293] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 08:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:59:58.721117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b2e66-79d9-4249-b597-bab1a085556f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:58.721696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b2e66-79d9-4249-b597-bab1a085556f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:58.721696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b2e66-79d9-4249-b597-bab1a085556f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:58.739413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2f8b2e66-79d9-4249-b597-bab1a085556f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:59:58.740662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2f8b2e66-79d9-4249-b597-bab1a085556f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:59:58.772710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:58.815010 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 643/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 08:59:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:59:58.837956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add8b9d5-d7fe-4103-8924-8c603cd6568f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:59:58.870103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:59:58.957605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add8b9d5-d7fe-4103-8924-8c603cd6568f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:59:58.992323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4023a3-0965-4a41-9058-eaae80183638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:59:58.992907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4023a3-0965-4a41-9058-eaae80183638 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:59:58.993192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4023a3-0965-4a41-9058-eaae80183638 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:59:58.993470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4023a3-0965-4a41-9058-eaae80183638 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:59:58.994215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4023a3-0965-4a41-9058-eaae80183638 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:59:58.995597 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4023a3-0965-4a41-9058-eaae80183638 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:59:59.008745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 652/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jul 21 08:59:58 2022] POST /identity/v3/services => generated 246 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:00:01.039170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295b529-7eb7-4aaf-a378-06675f614b94 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:00:01.039684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295b529-7eb7-4aaf-a378-06675f614b94 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:00:01.039684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295b529-7eb7-4aaf-a378-06675f614b94 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:00:01.041040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 644/1296] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:00:01.046406 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcba638-b991-49b4-a670-cb6f467e6cef None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:00:01.046860 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcba638-b991-49b4-a670-cb6f467e6cef None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:00:01.046994 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcba638-b991-49b4-a670-cb6f467e6cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:00:01.064601 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6fcba638-b991-49b4-a670-cb6f467e6cef None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:00:01.065809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6fcba638-b991-49b4-a670-cb6f467e6cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:00:01.140285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 653/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:00:01 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 09:00:01.162943 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295b529-7eb7-4aaf-a378-06675f614b94 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:00:01.265087 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295b529-7eb7-4aaf-a378-06675f614b94 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:00:01.301200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a4a8d-7c96-4265-b5e3-2a433e27dfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:00:01.301914 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a4a8d-7c96-4265-b5e3-2a433e27dfca None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:00:01.302201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a4a8d-7c96-4265-b5e3-2a433e27dfca None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:00:01.302744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a4a8d-7c96-4265-b5e3-2a433e27dfca None admin] PATH_INFO: `/v3/services/placement` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:00:01.303405 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a4a8d-7c96-4265-b5e3-2a433e27dfca None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:00:01.304838 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a4a8d-7c96-4265-b5e3-2a433e27dfca None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-808a4a8d-7c96-4265-b5e3-2a433e27dfca None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 09:00:01.317563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 09:00:01.318747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 09:00:01.319781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 09:00:01.319781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 09:00:01.319781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 09:00:01.319781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 09:00:01.319781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 09:00:01.319781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 09:00:01.319781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 09:00:01.319781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 09:00:01.319781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 09:00:01.319781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 09:00:01.319781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 09:00:01.319781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 09:00:01.319781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 09:00:01.319781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 09:00:01.319781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 645/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jul 21 09:00:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 09:00:01.324842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcba638-b991-49b4-a670-cb6f467e6cef None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:00:01.343935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:00:01.432770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcba638-b991-49b4-a670-cb6f467e6cef None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:00:01.467178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3367b-8dad-421b-8205-faf95265e36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:00:01.467838 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3367b-8dad-421b-8205-faf95265e36a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:00:01.468161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3367b-8dad-421b-8205-faf95265e36a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:00:01.468444 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3367b-8dad-421b-8205-faf95265e36a None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:00:01.469202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3367b-8dad-421b-8205-faf95265e36a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:00:01.469511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3367b-8dad-421b-8205-faf95265e36a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:00:01.470888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3367b-8dad-421b-8205-faf95265e36a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:00:01.476037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 654/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Jul 21 09:00:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:00:01.482125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a4a8d-7c96-4265-b5e3-2a433e27dfca None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:00:01.514275 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:00:01.583521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a4a8d-7c96-4265-b5e3-2a433e27dfca None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:00:01.616576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f136ba-a477-49f5-88b5-8452c37d4ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:00:01.617244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f136ba-a477-49f5-88b5-8452c37d4ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:00:01.617532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f136ba-a477-49f5-88b5-8452c37d4ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:00:01.617814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f136ba-a477-49f5-88b5-8452c37d4ff7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:00:01.618616 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f136ba-a477-49f5-88b5-8452c37d4ff7 None admin] RBAC: Adding query filter params (interface=public, service_id=45495a5757dc440cbb66e36c6745131d, region_id=RegionOne) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:00:01.618932 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f136ba-a477-49f5-88b5-8452c37d4ff7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:00:01.620363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f136ba-a477-49f5-88b5-8452c37d4ff7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:00:01.626377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 646/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jul 21 09:00:01 2022] GET /identity/v3/endpoints?service_id=45495a5757dc440cbb66e36c6745131d&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:00:01.632150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3367b-8dad-421b-8205-faf95265e36a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:00:01.716678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:00:01.737420 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3367b-8dad-421b-8205-faf95265e36a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:00:01.771905 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55012650-64e6-4738-a072-c2e273a8bd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:00:01.772585 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55012650-64e6-4738-a072-c2e273a8bd55 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:00:01.772870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55012650-64e6-4738-a072-c2e273a8bd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:00:01.773143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55012650-64e6-4738-a072-c2e273a8bd55 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:00:01.773827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55012650-64e6-4738-a072-c2e273a8bd55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:00:01.775193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55012650-64e6-4738-a072-c2e273a8bd55 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:00:01.782530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 655/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jul 21 09:00:01 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:00:03.850308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe926d2-fc7c-49cc-bb62-5961c94666ff None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:00:03.850854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe926d2-fc7c-49cc-bb62-5961c94666ff None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:00:03.850854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe926d2-fc7c-49cc-bb62-5961c94666ff None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:00:03.852279 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 647/1302] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:00:03.857569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3494a6a-1b2b-43cb-9e33-098cf011813f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:00:03.858008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3494a6a-1b2b-43cb-9e33-098cf011813f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:00:03.858169 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3494a6a-1b2b-43cb-9e33-098cf011813f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:00:03.874907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f3494a6a-1b2b-43cb-9e33-098cf011813f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:00:03.876132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f3494a6a-1b2b-43cb-9e33-098cf011813f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:00:03.944986 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 656/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:00:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 09:00:03.969576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe926d2-fc7c-49cc-bb62-5961c94666ff None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:00:04.065996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:00:04.073387 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe926d2-fc7c-49cc-bb62-5961c94666ff None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:00:04.107653 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648b43e-f698-4108-a08c-2e3e302532b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:00:04.108474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648b43e-f698-4108-a08c-2e3e302532b7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:00:04.108761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648b43e-f698-4108-a08c-2e3e302532b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:00:04.109051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648b43e-f698-4108-a08c-2e3e302532b7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:00:04.109770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648b43e-f698-4108-a08c-2e3e302532b7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:00:04.111151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648b43e-f698-4108-a08c-2e3e302532b7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-7648b43e-f698-4108-a08c-2e3e302532b7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 09:00:04.122977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 09:00:04.124152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 09:00:04.125634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 09:00:04.125634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 21 09:00:04.125634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 09:00:04.125634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 21 09:00:04.125634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 09:00:04.125634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 09:00:04.125634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 09:00:04.125634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 21 09:00:04.125634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 09:00:04.125634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 21 09:00:04.125634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 09:00:04.125634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 09:00:04.125634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 09:00:04.125634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 21 09:00:04.125634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 648/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jul 21 09:00:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 09:00:04.130818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3494a6a-1b2b-43cb-9e33-098cf011813f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:00:04.206068 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:00:04.228504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3494a6a-1b2b-43cb-9e33-098cf011813f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:00:04.262108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8e6b4-e38c-4624-99fb-94250aa8d6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:00:04.262763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8e6b4-e38c-4624-99fb-94250aa8d6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:00:04.263048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8e6b4-e38c-4624-99fb-94250aa8d6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:00:04.263317 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8e6b4-e38c-4624-99fb-94250aa8d6f4 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:00:04.264044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8e6b4-e38c-4624-99fb-94250aa8d6f4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:00:04.264353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8e6b4-e38c-4624-99fb-94250aa8d6f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:00:04.266287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8e6b4-e38c-4624-99fb-94250aa8d6f4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:00:04.271433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 657/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Jul 21 09:00:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:00:04.277187 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648b43e-f698-4108-a08c-2e3e302532b7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:00:04.378973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648b43e-f698-4108-a08c-2e3e302532b7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:00:04.413112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e4f9b-92df-479e-8b6c-20fbbb09f070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:00:04.413785 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e4f9b-92df-479e-8b6c-20fbbb09f070 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:00:04.414074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e4f9b-92df-479e-8b6c-20fbbb09f070 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:00:04.414354 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e4f9b-92df-479e-8b6c-20fbbb09f070 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:00:04.415107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e4f9b-92df-479e-8b6c-20fbbb09f070 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:00:04.416534 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e4f9b-92df-479e-8b6c-20fbbb09f070 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:00:04.436789 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 649/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jul 21 09:00:04 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:00:14.268395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=30 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:00:14.432397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:13.872836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b6661-81e1-4167-b26b-cd8090a20917 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:13.873967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b6661-81e1-4167-b26b-cd8090a20917 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:13.874675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b6661-81e1-4167-b26b-cd8090a20917 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:13.876305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 658/1307] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:01:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:01:13.882212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354406ad-c44f-4106-9251-3c911a58d2ef None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:13.882756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354406ad-c44f-4106-9251-3c911a58d2ef None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:13.883010 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354406ad-c44f-4106-9251-3c911a58d2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:13.964758 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-354406ad-c44f-4106-9251-3c911a58d2ef None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:01:13.965991 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-354406ad-c44f-4106-9251-3c911a58d2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:14.094843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 650/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:01:14.167641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857303c-d1fc-4fd5-946b-90e6088757e4 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:14.168497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857303c-d1fc-4fd5-946b-90e6088757e4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:14.169162 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857303c-d1fc-4fd5-946b-90e6088757e4 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:14.172313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 659/1309] 10.222.0.13 () {54 vars in 965 bytes} [Thu Jul 21 09:01:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:01:14.180814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8261d77-86b4-4a61-9e1a-96167027e825 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:14.181151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8261d77-86b4-4a61-9e1a-96167027e825 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:14.181618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8261d77-86b4-4a61-9e1a-96167027e825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:14.312267 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f8261d77-86b4-4a61-9e1a-96167027e825 None None] MFA Rules not processed for user `9e2a13c2cb864175a082de3ca8b166c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:01:14.313668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f8261d77-86b4-4a61-9e1a-96167027e825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:14.339307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:14.417785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857303c-d1fc-4fd5-946b-90e6088757e4 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:01:14.419707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6857303c-d1fc-4fd5-946b-90e6088757e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:14.424988 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 651/1310] 10.222.0.13 () {58 vars in 1062 bytes} [Thu Jul 21 09:01:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:01:14.613017 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857303c-d1fc-4fd5-946b-90e6088757e4 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:01:14.706859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a68f0-86d2-4fbf-bc64-83fe45b8d27f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:01:14.707519 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a68f0-86d2-4fbf-bc64-83fe45b8d27f service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:14.707798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a68f0-86d2-4fbf-bc64-83fe45b8d27f service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:14.708097 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a68f0-86d2-4fbf-bc64-83fe45b8d27f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:14.709796 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b27a68f0-86d2-4fbf-bc64-83fe45b8d27f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:14.770995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:14.863797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a68f0-86d2-4fbf-bc64-83fe45b8d27f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:01:14.866700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a68f0-86d2-4fbf-bc64-83fe45b8d27f service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:01:14.906573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 660/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:01:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:01:17.462076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df357e-a8c6-4489-982d-7971efa740b2 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:17.462648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df357e-a8c6-4489-982d-7971efa740b2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:17.462648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1df357e-a8c6-4489-982d-7971efa740b2 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:17.464440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 652/1312] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:01:17.470450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31129f7a-abef-4ccf-9f3e-db1977f34de2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:17.471280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31129f7a-abef-4ccf-9f3e-db1977f34de2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:17.471280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31129f7a-abef-4ccf-9f3e-db1977f34de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:17.619727 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-31129f7a-abef-4ccf-9f3e-db1977f34de2 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:01:17.621719 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-31129f7a-abef-4ccf-9f3e-db1977f34de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:17.782390 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 661/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:01:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:01:17.835120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704bb8d-bba4-40a5-a6a9-ff0fd28b94ed None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:17.835509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704bb8d-bba4-40a5-a6a9-ff0fd28b94ed None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:17.835852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704bb8d-bba4-40a5-a6a9-ff0fd28b94ed None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:17.837359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 653/1314] 10.222.0.13 () {54 vars in 965 bytes} [Thu Jul 21 09:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:01:17.843751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5173098-78e6-4cb5-8e4e-998131d591f8 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:17.844153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5173098-78e6-4cb5-8e4e-998131d591f8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:17.844502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5173098-78e6-4cb5-8e4e-998131d591f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:17.856157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:17.927838 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a5173098-78e6-4cb5-8e4e-998131d591f8 None None] MFA Rules not processed for user `9e2a13c2cb864175a082de3ca8b166c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:01:17.929375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a5173098-78e6-4cb5-8e4e-998131d591f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:18.018133 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 662/1315] 10.222.0.13 () {58 vars in 1062 bytes} [Thu Jul 21 09:01:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:01:18.027326 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704bb8d-bba4-40a5-a6a9-ff0fd28b94ed None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:01:18.028870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f704bb8d-bba4-40a5-a6a9-ff0fd28b94ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:18.142551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704bb8d-bba4-40a5-a6a9-ff0fd28b94ed None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:01:18.247786 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:18.253899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0753eb5-6fb1-4227-9fb0-37d321144838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:01:18.255173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0753eb5-6fb1-4227-9fb0-37d321144838 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:18.255714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0753eb5-6fb1-4227-9fb0-37d321144838 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:18.256288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0753eb5-6fb1-4227-9fb0-37d321144838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:18.260032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c0753eb5-6fb1-4227-9fb0-37d321144838 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:18.381794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0753eb5-6fb1-4227-9fb0-37d321144838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:01:18.383247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0753eb5-6fb1-4227-9fb0-37d321144838 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:01:18.402851 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 654/1316] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:01:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:01:21.067877 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d1afb-6cae-489a-8071-78dbdf13c0f7 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:21.068474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d1afb-6cae-489a-8071-78dbdf13c0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:21.068735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d1afb-6cae-489a-8071-78dbdf13c0f7 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:21.070306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 663/1317] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:01:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:01:21.076147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1078f05-7918-4a5f-9a05-936e5d0e4e29 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:21.076626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1078f05-7918-4a5f-9a05-936e5d0e4e29 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:21.076878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1078f05-7918-4a5f-9a05-936e5d0e4e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:21.144912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f1078f05-7918-4a5f-9a05-936e5d0e4e29 None None] MFA Rules not processed for user `e430ce5b82254dcb8def7ae26a6c6e27`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:01:21.146134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f1078f05-7918-4a5f-9a05-936e5d0e4e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:21.200313 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:21.247955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 655/1318] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:01:21 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:01:21.289118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d1afb-6cae-489a-8071-78dbdf13c0f7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:01:21.290639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5a8d1afb-6cae-489a-8071-78dbdf13c0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:21.400842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d1afb-6cae-489a-8071-78dbdf13c0f7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:01:21.403841 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:21.480707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478411af-15f0-4635-b668-2f67813c6bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:01:21.481364 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478411af-15f0-4635-b668-2f67813c6bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:21.481651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478411af-15f0-4635-b668-2f67813c6bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:21.481930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478411af-15f0-4635-b668-2f67813c6bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:21.484410 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-478411af-15f0-4635-b668-2f67813c6bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:21.598181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478411af-15f0-4635-b668-2f67813c6bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:01:21.599641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478411af-15f0-4635-b668-2f67813c6bc6 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:01:21.618254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 664/1319] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:01:21 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:01:24.317013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff29ec-59c8-427a-aaf9-db1fe2788697 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:24.317604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff29ec-59c8-427a-aaf9-db1fe2788697 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:24.317604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff29ec-59c8-427a-aaf9-db1fe2788697 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:24.319124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 656/1320] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:01:24.324427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ac519-1bba-4a9d-baa0-3d840e6de586 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:24.324832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ac519-1bba-4a9d-baa0-3d840e6de586 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:24.324985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ac519-1bba-4a9d-baa0-3d840e6de586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:24.364220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:24.390850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-548ac519-1bba-4a9d-baa0-3d840e6de586 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:01:24.392122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-548ac519-1bba-4a9d-baa0-3d840e6de586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:24.489228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 665/1321] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:01:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:01:24.534443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff29ec-59c8-427a-aaf9-db1fe2788697 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:01:24.535987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4bff29ec-59c8-427a-aaf9-db1fe2788697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:24.639584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff29ec-59c8-427a-aaf9-db1fe2788697 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:01:24.721476 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f7ffa-281a-47b1-be74-85c32d01dd59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:01:24.722151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f7ffa-281a-47b1-be74-85c32d01dd59 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:24.722432 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f7ffa-281a-47b1-be74-85c32d01dd59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:24.722705 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f7ffa-281a-47b1-be74-85c32d01dd59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:24.724501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6c2f7ffa-281a-47b1-be74-85c32d01dd59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:24.733773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:24.851490 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f7ffa-281a-47b1-be74-85c32d01dd59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:01:24.853218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f7ffa-281a-47b1-be74-85c32d01dd59 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:01:24.871994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 657/1322] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:01:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:01:27.820317 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed5172-7117-4dd6-b09f-b0850edec82e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:27.820939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed5172-7117-4dd6-b09f-b0850edec82e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:27.820939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed5172-7117-4dd6-b09f-b0850edec82e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:27.822266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 666/1323] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:01:27.828126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebde953-9836-4adc-ae2c-23184198ddd5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:27.828596 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebde953-9836-4adc-ae2c-23184198ddd5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:27.828707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebde953-9836-4adc-ae2c-23184198ddd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:27.899239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2ebde953-9836-4adc-ae2c-23184198ddd5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:01:27.900520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2ebde953-9836-4adc-ae2c-23184198ddd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:27.972059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:28.020439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 658/1324] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:01:28.058023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed5172-7117-4dd6-b09f-b0850edec82e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:01:28.059609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8fed5172-7117-4dd6-b09f-b0850edec82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:28.174003 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed5172-7117-4dd6-b09f-b0850edec82e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:01:28.186714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:28.252544 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d8699-5376-4af4-a29b-c66d6872223a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:01:28.253200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d8699-5376-4af4-a29b-c66d6872223a service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:28.253485 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d8699-5376-4af4-a29b-c66d6872223a service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:28.253761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d8699-5376-4af4-a29b-c66d6872223a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:28.255411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-239d8699-5376-4af4-a29b-c66d6872223a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:28.373671 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d8699-5376-4af4-a29b-c66d6872223a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:01:28.374533 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d8699-5376-4af4-a29b-c66d6872223a service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:01:28.393253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 667/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:01:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:01:30.564106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955702e-23fe-4331-8d98-0fb4abc51ff7 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:30.564628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955702e-23fe-4331-8d98-0fb4abc51ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:30.564628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955702e-23fe-4331-8d98-0fb4abc51ff7 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:30.566044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 659/1326] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:01:30.571602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68a1b1-984e-4fb1-8f82-53dc06cc3f8b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:30.572193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68a1b1-984e-4fb1-8f82-53dc06cc3f8b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:30.572193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68a1b1-984e-4fb1-8f82-53dc06cc3f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:30.624132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:30.651571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-fb68a1b1-984e-4fb1-8f82-53dc06cc3f8b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:01:30.652808 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fb68a1b1-984e-4fb1-8f82-53dc06cc3f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:30.771195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 668/1327] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:01:30.809515 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955702e-23fe-4331-8d98-0fb4abc51ff7 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:01:30.812375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0955702e-23fe-4331-8d98-0fb4abc51ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:30.935936 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955702e-23fe-4331-8d98-0fb4abc51ff7 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:01:31.017270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa1383-16de-4700-91bf-6ae27ad9b854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:01:31.018057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa1383-16de-4700-91bf-6ae27ad9b854 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:31.018406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa1383-16de-4700-91bf-6ae27ad9b854 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:31.018752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa1383-16de-4700-91bf-6ae27ad9b854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:31.022965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-74fa1383-16de-4700-91bf-6ae27ad9b854 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:31.048183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:31.157660 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa1383-16de-4700-91bf-6ae27ad9b854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:01:31.159156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa1383-16de-4700-91bf-6ae27ad9b854 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:01:31.181448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 660/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:01:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:01:33.338496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e52d1-e38b-457c-871a-08a0a9ee88d9 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:33.339134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e52d1-e38b-457c-871a-08a0a9ee88d9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:33.339134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e52d1-e38b-457c-871a-08a0a9ee88d9 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:33.340448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 669/1329] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:01:33.345767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d5660-012e-4907-9145-c036d40a91a9 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:33.346210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d5660-012e-4907-9145-c036d40a91a9 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:33.346317 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d5660-012e-4907-9145-c036d40a91a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:33.413670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e02d5660-012e-4907-9145-c036d40a91a9 None None] MFA Rules not processed for user `e430ce5b82254dcb8def7ae26a6c6e27`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:01:33.414873 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e02d5660-012e-4907-9145-c036d40a91a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:33.487370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:33.525330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 661/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:01:33 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:01:33.565153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e52d1-e38b-457c-871a-08a0a9ee88d9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:01:33.566705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d18e52d1-e38b-457c-871a-08a0a9ee88d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:33.694414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e52d1-e38b-457c-871a-08a0a9ee88d9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:01:33.723706 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:33.792638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841279f-4a6c-4686-be4b-bc023d64c94a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:01:33.793295 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841279f-4a6c-4686-be4b-bc023d64c94a service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:33.793577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841279f-4a6c-4686-be4b-bc023d64c94a service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:33.793853 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841279f-4a6c-4686-be4b-bc023d64c94a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:33.795586 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f841279f-4a6c-4686-be4b-bc023d64c94a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:33.914792 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841279f-4a6c-4686-be4b-bc023d64c94a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:01:33.916308 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841279f-4a6c-4686-be4b-bc023d64c94a service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:01:33.936628 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 670/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:01:33 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:01:36.430600 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedcb55-4c9c-4215-ab99-40b964b5fdd0 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:36.431175 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedcb55-4c9c-4215-ab99-40b964b5fdd0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:36.431175 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedcb55-4c9c-4215-ab99-40b964b5fdd0 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:36.432542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 662/1332] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:01:36.437975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824ab1a-a13a-4645-b21e-20d67f15e60e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:36.438403 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824ab1a-a13a-4645-b21e-20d67f15e60e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:36.438545 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824ab1a-a13a-4645-b21e-20d67f15e60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:36.504119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:36.507133 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f824ab1a-a13a-4645-b21e-20d67f15e60e None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:01:36.508938 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f824ab1a-a13a-4645-b21e-20d67f15e60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:36.609262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 671/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:01:36.646270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedcb55-4c9c-4215-ab99-40b964b5fdd0 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:01:36.647826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fdedcb55-4c9c-4215-ab99-40b964b5fdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:36.753555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedcb55-4c9c-4215-ab99-40b964b5fdd0 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:01:36.827496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60086ef-464a-40f4-8cb9-b388cb8f0dcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:01:36.828172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60086ef-464a-40f4-8cb9-b388cb8f0dcd service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:36.828455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60086ef-464a-40f4-8cb9-b388cb8f0dcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:36.828727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60086ef-464a-40f4-8cb9-b388cb8f0dcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:36.830380 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d60086ef-464a-40f4-8cb9-b388cb8f0dcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:36.860199 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:36.952566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60086ef-464a-40f4-8cb9-b388cb8f0dcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:01:36.954037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60086ef-464a-40f4-8cb9-b388cb8f0dcd service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:01:36.975151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 663/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:01:39.103101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85f0a6-a3df-46f5-a1fb-050650fe33df None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:39.103637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85f0a6-a3df-46f5-a1fb-050650fe33df None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:39.103637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85f0a6-a3df-46f5-a1fb-050650fe33df None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:39.104995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 672/1335] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:01:39.111154 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7a83a-8223-412b-9a00-d62811eb58f6 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:39.111604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7a83a-8223-412b-9a00-d62811eb58f6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:39.111712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7a83a-8223-412b-9a00-d62811eb58f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:39.181902 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f8d7a83a-8223-412b-9a00-d62811eb58f6 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:01:39.183135 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f8d7a83a-8223-412b-9a00-d62811eb58f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:39.269213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:39.295693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 664/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:01:39.338639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85f0a6-a3df-46f5-a1fb-050650fe33df None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:01:39.340188 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ad85f0a6-a3df-46f5-a1fb-050650fe33df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:39.444091 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85f0a6-a3df-46f5-a1fb-050650fe33df None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:01:39.472095 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:39.517979 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecb6ef-64b9-4602-9b11-10136d36514f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:01:39.518621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecb6ef-64b9-4602-9b11-10136d36514f service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:39.518903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecb6ef-64b9-4602-9b11-10136d36514f service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:39.519176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecb6ef-64b9-4602-9b11-10136d36514f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:39.520839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-53ecb6ef-64b9-4602-9b11-10136d36514f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:39.629692 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecb6ef-64b9-4602-9b11-10136d36514f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:01:39.631143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecb6ef-64b9-4602-9b11-10136d36514f service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:01:39.650009 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 673/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:01:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:01:42.441913 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966990e-4add-4b37-8ff3-15313a44ef50 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:42.442465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966990e-4add-4b37-8ff3-15313a44ef50 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:42.442465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966990e-4add-4b37-8ff3-15313a44ef50 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:42.443852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 665/1338] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:01:42.449753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe02e3c-4e10-40a9-ac5f-fe5085d61882 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:42.450284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe02e3c-4e10-40a9-ac5f-fe5085d61882 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:42.451116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe02e3c-4e10-40a9-ac5f-fe5085d61882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:42.525003 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-efe02e3c-4e10-40a9-ac5f-fe5085d61882 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:01:42.526256 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-efe02e3c-4e10-40a9-ac5f-fe5085d61882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:42.537268 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:42.630651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 674/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:01:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:01:42.667702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966990e-4add-4b37-8ff3-15313a44ef50 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:01:42.669980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5966990e-4add-4b37-8ff3-15313a44ef50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:42.778305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966990e-4add-4b37-8ff3-15313a44ef50 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:01:42.851803 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b415960-1615-445e-85c2-f8a95da8e12c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:01:42.852474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b415960-1615-445e-85c2-f8a95da8e12c service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:42.852755 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b415960-1615-445e-85c2-f8a95da8e12c service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:42.853026 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b415960-1615-445e-85c2-f8a95da8e12c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:42.854655 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1b415960-1615-445e-85c2-f8a95da8e12c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:42.891353 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:42.974040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b415960-1615-445e-85c2-f8a95da8e12c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:01:42.975406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b415960-1615-445e-85c2-f8a95da8e12c service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:01:42.995182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 666/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:01:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:01:47.462421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d370a11-dafc-404c-836c-48b0fe6b7bd9 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:47.463004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d370a11-dafc-404c-836c-48b0fe6b7bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:47.463004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d370a11-dafc-404c-836c-48b0fe6b7bd9 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:47.464415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 675/1341] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:01:47.470558 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c0e4d-8eb2-471c-9d05-40ec761ad940 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:47.471098 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c0e4d-8eb2-471c-9d05-40ec761ad940 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:47.471098 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c0e4d-8eb2-471c-9d05-40ec761ad940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:47.539100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6c4c0e4d-8eb2-471c-9d05-40ec761ad940 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:01:47.540343 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6c4c0e4d-8eb2-471c-9d05-40ec761ad940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:47.633992 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:47.645389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 667/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:01:47.686169 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d370a11-dafc-404c-836c-48b0fe6b7bd9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:01:47.687702 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3d370a11-dafc-404c-836c-48b0fe6b7bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:47.801438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d370a11-dafc-404c-836c-48b0fe6b7bd9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:01:47.839851 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:47.876959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14919b9a-ac3c-412a-b889-1021b3968121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:01:47.877605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14919b9a-ac3c-412a-b889-1021b3968121 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:47.877889 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14919b9a-ac3c-412a-b889-1021b3968121 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:47.878168 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14919b9a-ac3c-412a-b889-1021b3968121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:47.879900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-14919b9a-ac3c-412a-b889-1021b3968121 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:47.994806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14919b9a-ac3c-412a-b889-1021b3968121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:01:47.996306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14919b9a-ac3c-412a-b889-1021b3968121 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:01:48.015000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 676/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:01:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:01:50.720382 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e9acb-411c-4cda-9f91-831a52a4e540 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:50.720927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e9acb-411c-4cda-9f91-831a52a4e540 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:50.720927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e9acb-411c-4cda-9f91-831a52a4e540 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:50.722553 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 668/1344] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:01:50.728448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf13a70-4235-4b30-881d-cf6a0c1b4a3f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:50.729042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf13a70-4235-4b30-881d-cf6a0c1b4a3f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:50.729042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf13a70-4235-4b30-881d-cf6a0c1b4a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:50.804063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-bdf13a70-4235-4b30-881d-cf6a0c1b4a3f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:01:50.805294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-bdf13a70-4235-4b30-881d-cf6a0c1b4a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:50.826825 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:50.918764 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 677/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:01:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:01:50.959825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e9acb-411c-4cda-9f91-831a52a4e540 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:01:50.961452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8e1e9acb-411c-4cda-9f91-831a52a4e540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:51.083811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e9acb-411c-4cda-9f91-831a52a4e540 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:01:51.163569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503f139-d1ca-4aef-9793-6d7c75dd03bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:01:51.164259 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503f139-d1ca-4aef-9793-6d7c75dd03bc service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:51.164552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503f139-d1ca-4aef-9793-6d7c75dd03bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:51.164827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503f139-d1ca-4aef-9793-6d7c75dd03bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:51.166500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7503f139-d1ca-4aef-9793-6d7c75dd03bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:51.215478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:51.286554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503f139-d1ca-4aef-9793-6d7c75dd03bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:01:51.288054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503f139-d1ca-4aef-9793-6d7c75dd03bc service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:01:51.308825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 669/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:01:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:01:55.518183 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83564a7-cd48-4f37-8113-c2cd182e5b3e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:55.518698 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83564a7-cd48-4f37-8113-c2cd182e5b3e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:55.519319 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83564a7-cd48-4f37-8113-c2cd182e5b3e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:55.522122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 678/1347] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:01:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:01:55.532066 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9474eae-d4da-4c64-9818-a3576a33f349 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:55.532577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9474eae-d4da-4c64-9818-a3576a33f349 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:55.533080 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9474eae-d4da-4c64-9818-a3576a33f349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:55.638895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d9474eae-d4da-4c64-9818-a3576a33f349 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:01:55.640160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d9474eae-d4da-4c64-9818-a3576a33f349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:55.785106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:55.796319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 670/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:01:55.832607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83564a7-cd48-4f37-8113-c2cd182e5b3e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:01:55.834228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f83564a7-cd48-4f37-8113-c2cd182e5b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:55.962735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83564a7-cd48-4f37-8113-c2cd182e5b3e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:01:56.017829 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:01:56.048872 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabe24a-ce0c-48d7-9992-96dc9121dfec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:01:56.049598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabe24a-ce0c-48d7-9992-96dc9121dfec service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:01:56.049889 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabe24a-ce0c-48d7-9992-96dc9121dfec service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:01:56.050171 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabe24a-ce0c-48d7-9992-96dc9121dfec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:01:56.051866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8fabe24a-ce0c-48d7-9992-96dc9121dfec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:01:56.176002 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabe24a-ce0c-48d7-9992-96dc9121dfec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:01:56.177461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabe24a-ce0c-48d7-9992-96dc9121dfec service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:01:56.199764 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 679/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:01:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:02:01.242352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a431e24-b39d-4fdb-9162-954dffbee715 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:01.242804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a431e24-b39d-4fdb-9162-954dffbee715 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:01.242913 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a431e24-b39d-4fdb-9162-954dffbee715 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:01.244338 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 671/1350] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jul 21 09:02:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:01.250482 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9a53e-578b-459f-83d8-5420532c90be None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:01.251020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9a53e-578b-459f-83d8-5420532c90be None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:01.251020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9a53e-578b-459f-83d8-5420532c90be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:01.371261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-80f9a53e-578b-459f-83d8-5420532c90be None None] MFA Rules not processed for user `72f9ef6de66e456e800237e9f954e98d`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:01.372826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-80f9a53e-578b-459f-83d8-5420532c90be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:01.402033 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:01.556504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 680/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jul 21 09:02:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:06.024295 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f100b6d-f30b-4ca6-92ae-3e944db3e900 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:06.025137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f100b6d-f30b-4ca6-92ae-3e944db3e900 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:06.025589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f100b6d-f30b-4ca6-92ae-3e944db3e900 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:06.027174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 672/1352] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jul 21 09:02:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:06.033666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052f9f1-1183-4143-a2d6-4aba7b6b1e7e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:06.034139 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052f9f1-1183-4143-a2d6-4aba7b6b1e7e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:06.034220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052f9f1-1183-4143-a2d6-4aba7b6b1e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:06.115708 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-2052f9f1-1183-4143-a2d6-4aba7b6b1e7e None None] MFA Rules not processed for user `72f9ef6de66e456e800237e9f954e98d`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:06.117057 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2052f9f1-1183-4143-a2d6-4aba7b6b1e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:06.203771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:06.246295 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 681/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jul 21 09:02:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:07.451403 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e8be4-d621-4b29-9d0c-3febe0a55d02 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:07.451975 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e8be4-d621-4b29-9d0c-3febe0a55d02 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:07.451975 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e8be4-d621-4b29-9d0c-3febe0a55d02 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:07.453369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 673/1354] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jul 21 09:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:07.459077 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1eeb69-6b5e-4dbf-b892-39b5c6c17462 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:07.459609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1eeb69-6b5e-4dbf-b892-39b5c6c17462 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:07.459710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1eeb69-6b5e-4dbf-b892-39b5c6c17462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:07.554716 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3c1eeb69-6b5e-4dbf-b892-39b5c6c17462 None None] MFA Rules not processed for user `72f9ef6de66e456e800237e9f954e98d`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:07.555976 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3c1eeb69-6b5e-4dbf-b892-39b5c6c17462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:07.687479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 682/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jul 21 09:02:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:02:11.251430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89d05b-923f-4faf-8026-6131927f7da6 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:11.252021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89d05b-923f-4faf-8026-6131927f7da6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:11.252021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89d05b-923f-4faf-8026-6131927f7da6 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:11.254322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 674/1356] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:02:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:11.259575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e32cd-d5c2-4278-aad1-4a7bdadffb8e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:11.260192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e32cd-d5c2-4278-aad1-4a7bdadffb8e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:11.260192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e32cd-d5c2-4278-aad1-4a7bdadffb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:11.301469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:11.343683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-542e32cd-d5c2-4278-aad1-4a7bdadffb8e None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:11.345099 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-542e32cd-d5c2-4278-aad1-4a7bdadffb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:11.461806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 683/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:11.519692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a497fcd-d50d-4f86-8121-65b8a9f6cb84 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:11.520055 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a497fcd-d50d-4f86-8121-65b8a9f6cb84 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:11.520276 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a497fcd-d50d-4f86-8121-65b8a9f6cb84 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:11.521835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 675/1358] 10.222.0.13 () {54 vars in 962 bytes} [Thu Jul 21 09:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:11.532854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db75dc2-908b-4c40-a381-9d60a98aecdb None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:11.533339 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db75dc2-908b-4c40-a381-9d60a98aecdb None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:11.533724 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db75dc2-908b-4c40-a381-9d60a98aecdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:11.628200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3db75dc2-908b-4c40-a381-9d60a98aecdb None None] MFA Rules not processed for user `c48d41dcf9b344a8b2fec77f4dd75fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:11.631893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3db75dc2-908b-4c40-a381-9d60a98aecdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:11.686883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:11.764930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 684/1359] 10.222.0.13 () {58 vars in 1059 bytes} [Thu Jul 21 09:02:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:11.772699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a497fcd-d50d-4f86-8121-65b8a9f6cb84 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:02:11.774191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7a497fcd-d50d-4f86-8121-65b8a9f6cb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:11.957620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a497fcd-d50d-4f86-8121-65b8a9f6cb84 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:02:12.039540 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:12.087126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a098df-8af8-49a2-a494-df04ce05e7c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:02:12.087778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a098df-8af8-49a2-a494-df04ce05e7c5 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:12.088101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a098df-8af8-49a2-a494-df04ce05e7c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:12.088377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a098df-8af8-49a2-a494-df04ce05e7c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:12.090007 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-00a098df-8af8-49a2-a494-df04ce05e7c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:12.193896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a098df-8af8-49a2-a494-df04ce05e7c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:02:12.195848 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a098df-8af8-49a2-a494-df04ce05e7c5 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:02:12.217619 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 676/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:02:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:02:14.672475 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b6971-ce5d-44a8-abae-2017cc47ef17 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:14.673072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b6971-ce5d-44a8-abae-2017cc47ef17 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:14.673072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b6971-ce5d-44a8-abae-2017cc47ef17 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:14.674562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 685/1361] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:14.680427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20018fd4-199a-48c6-89c1-5ecdd4689e13 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:14.680903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20018fd4-199a-48c6-89c1-5ecdd4689e13 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:14.681020 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20018fd4-199a-48c6-89c1-5ecdd4689e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:14.768968 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-20018fd4-199a-48c6-89c1-5ecdd4689e13 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:14.770366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-20018fd4-199a-48c6-89c1-5ecdd4689e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:14.829377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:14.914865 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 677/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:14.961760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b6971-ce5d-44a8-abae-2017cc47ef17 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:02:14.963650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-683b6971-ce5d-44a8-abae-2017cc47ef17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:15.175245 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b6971-ce5d-44a8-abae-2017cc47ef17 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:02:15.186391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:15.322308 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e606e6-c267-4761-955d-3bb2507bb5a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:02:15.323424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e606e6-c267-4761-955d-3bb2507bb5a0 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:15.323967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e606e6-c267-4761-955d-3bb2507bb5a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:15.324347 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e606e6-c267-4761-955d-3bb2507bb5a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:15.328173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-45e606e6-c267-4761-955d-3bb2507bb5a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:15.469849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e606e6-c267-4761-955d-3bb2507bb5a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:02:15.471499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e606e6-c267-4761-955d-3bb2507bb5a0 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:02:15.493603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 686/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:02:18.073499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1b9a4-4c88-4072-b2eb-b28ea74aba7c None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:18.074072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1b9a4-4c88-4072-b2eb-b28ea74aba7c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:18.074072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1b9a4-4c88-4072-b2eb-b28ea74aba7c None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:18.076480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 678/1364] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:18.082219 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e5ff0-eb12-453d-952b-bb24f8e2b5fe None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:18.082734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e5ff0-eb12-453d-952b-bb24f8e2b5fe None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:18.082734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e5ff0-eb12-453d-952b-bb24f8e2b5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:18.099209 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:18.159016 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0a2e5ff0-eb12-453d-952b-bb24f8e2b5fe None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:18.160473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0a2e5ff0-eb12-453d-952b-bb24f8e2b5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:18.272645 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 687/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:18.322363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f00a50-01b3-4294-a8c5-ed7fcad61af1 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:18.322997 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f00a50-01b3-4294-a8c5-ed7fcad61af1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:18.323279 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f00a50-01b3-4294-a8c5-ed7fcad61af1 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:18.325217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 679/1366] 10.222.0.13 () {54 vars in 962 bytes} [Thu Jul 21 09:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:18.331083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5c5b5-3d9a-4c6f-9d6e-43c691cc8b16 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:18.331377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5c5b5-3d9a-4c6f-9d6e-43c691cc8b16 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:18.331660 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5c5b5-3d9a-4c6f-9d6e-43c691cc8b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:18.415961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-9eb5c5b5-3d9a-4c6f-9d6e-43c691cc8b16 None None] MFA Rules not processed for user `c48d41dcf9b344a8b2fec77f4dd75fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:18.417113 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9eb5c5b5-3d9a-4c6f-9d6e-43c691cc8b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:18.458072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:18.547948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 688/1367] 10.222.0.13 () {58 vars in 1059 bytes} [Thu Jul 21 09:02:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:18.556257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f00a50-01b3-4294-a8c5-ed7fcad61af1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:02:18.557835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b3f00a50-01b3-4294-a8c5-ed7fcad61af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:18.758474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f00a50-01b3-4294-a8c5-ed7fcad61af1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:02:18.785322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:18.902495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4afbc-dccb-4700-9405-5166ba445fe2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:02:18.903367 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4afbc-dccb-4700-9405-5166ba445fe2 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:18.903793 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4afbc-dccb-4700-9405-5166ba445fe2 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:18.904217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4afbc-dccb-4700-9405-5166ba445fe2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:18.906035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7be4afbc-dccb-4700-9405-5166ba445fe2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:19.024472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4afbc-dccb-4700-9405-5166ba445fe2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:02:19.026351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4afbc-dccb-4700-9405-5166ba445fe2 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:02:19.051673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 680/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:02:19.712003 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11016c25-1fde-43ac-8967-60f4a7551298 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:19.712003 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11016c25-1fde-43ac-8967-60f4a7551298 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:19.712003 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11016c25-1fde-43ac-8967-60f4a7551298 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:19.712766 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 689/1369] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:19.717871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e73f89-7269-4bad-9cf3-9936bc35cb2f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:19.718317 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e73f89-7269-4bad-9cf3-9936bc35cb2f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:19.718475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e73f89-7269-4bad-9cf3-9936bc35cb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:19.751484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:19.792473 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e8e73f89-7269-4bad-9cf3-9936bc35cb2f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:19.793914 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e8e73f89-7269-4bad-9cf3-9936bc35cb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:19.906986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 681/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:20.221459 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b365d61-9b30-4268-8fd2-4342a2aeae8d None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:20.221939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b365d61-9b30-4268-8fd2-4342a2aeae8d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:20.222438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b365d61-9b30-4268-8fd2-4342a2aeae8d None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:20.224105 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 690/1371] 10.222.0.13 () {54 vars in 964 bytes} [Thu Jul 21 09:02:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:20.229397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48800a9e-a080-46d7-894e-4a5e57cbadff None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:20.229900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48800a9e-a080-46d7-894e-4a5e57cbadff None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:20.230185 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48800a9e-a080-46d7-894e-4a5e57cbadff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:20.319979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-48800a9e-a080-46d7-894e-4a5e57cbadff None None] MFA Rules not processed for user `8a3b3421e9d8427388f4fd2be5a601cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:20.321247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-48800a9e-a080-46d7-894e-4a5e57cbadff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:20.373546 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:20.410487 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 682/1372] 10.222.0.13 () {58 vars in 1061 bytes} [Thu Jul 21 09:02:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:20.418511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b365d61-9b30-4268-8fd2-4342a2aeae8d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:02:20.420237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3b365d61-9b30-4268-8fd2-4342a2aeae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:20.561084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b365d61-9b30-4268-8fd2-4342a2aeae8d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:02:20.607607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:20.654786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330e06d5-a3f7-47af-8e25-69b309bea68f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:02:20.655632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e06d5-a3f7-47af-8e25-69b309bea68f service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:20.656050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e06d5-a3f7-47af-8e25-69b309bea68f service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:20.656424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e06d5-a3f7-47af-8e25-69b309bea68f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:20.658386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-330e06d5-a3f7-47af-8e25-69b309bea68f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:20.805859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330e06d5-a3f7-47af-8e25-69b309bea68f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:02:20.807560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330e06d5-a3f7-47af-8e25-69b309bea68f service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:02:20.831453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 691/1373] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:02:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:02:21.222878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d9b03-12a2-4e72-acdf-78eb3105af81 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:21.223416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d9b03-12a2-4e72-acdf-78eb3105af81 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:21.223416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d9b03-12a2-4e72-acdf-78eb3105af81 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:21.224867 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 683/1374] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:21.230070 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef7ddb-7aa5-45e7-abc2-68af4bc74fd2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:21.230361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef7ddb-7aa5-45e7-abc2-68af4bc74fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:21.231292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef7ddb-7aa5-45e7-abc2-68af4bc74fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:21.300528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b9ef7ddb-7aa5-45e7-abc2-68af4bc74fd2 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:21.302058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b9ef7ddb-7aa5-45e7-abc2-68af4bc74fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:21.323900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:21.407972 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 692/1375] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:21.454369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d9b03-12a2-4e72-acdf-78eb3105af81 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:02:21.455983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ec6d9b03-12a2-4e72-acdf-78eb3105af81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:21.617615 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d9b03-12a2-4e72-acdf-78eb3105af81 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:02:21.680737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:21.748016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77010db8-30f4-425a-8717-dd6c7eb5137f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:02:21.748669 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77010db8-30f4-425a-8717-dd6c7eb5137f service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:21.748954 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77010db8-30f4-425a-8717-dd6c7eb5137f service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:21.749229 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77010db8-30f4-425a-8717-dd6c7eb5137f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:21.750896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-77010db8-30f4-425a-8717-dd6c7eb5137f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:21.868950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77010db8-30f4-425a-8717-dd6c7eb5137f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:02:21.870511 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77010db8-30f4-425a-8717-dd6c7eb5137f service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:02:21.889870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 684/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:02:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:02:24.368067 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b0df2-c3df-4fc8-86eb-46559cefec64 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:24.368577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b0df2-c3df-4fc8-86eb-46559cefec64 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:24.368700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b0df2-c3df-4fc8-86eb-46559cefec64 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:24.370251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 693/1377] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:24.377637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43dea70-21f6-4ba2-b90f-eee1ffe5ffb9 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:24.378122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43dea70-21f6-4ba2-b90f-eee1ffe5ffb9 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:24.378214 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43dea70-21f6-4ba2-b90f-eee1ffe5ffb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:24.466344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b43dea70-21f6-4ba2-b90f-eee1ffe5ffb9 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:24.467998 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b43dea70-21f6-4ba2-b90f-eee1ffe5ffb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:24.477107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:24.617877 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 685/1378] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:02:24.663757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b0df2-c3df-4fc8-86eb-46559cefec64 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:02:24.665350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-163b0df2-c3df-4fc8-86eb-46559cefec64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:24.846741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b0df2-c3df-4fc8-86eb-46559cefec64 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:02:24.851167 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:24.984945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef2158-0d7a-44de-9ec6-e82c1d7f3a14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:02:24.985728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef2158-0d7a-44de-9ec6-e82c1d7f3a14 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:24.985728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef2158-0d7a-44de-9ec6-e82c1d7f3a14 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:24.986046 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef2158-0d7a-44de-9ec6-e82c1d7f3a14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:24.987653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1eef2158-0d7a-44de-9ec6-e82c1d7f3a14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:25.087010 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef2158-0d7a-44de-9ec6-e82c1d7f3a14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:02:25.088652 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef2158-0d7a-44de-9ec6-e82c1d7f3a14 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:02:25.114772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 694/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:02:27.230549 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60055553-47ec-43d8-bf96-8b8f95cf0b42 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:27.231171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60055553-47ec-43d8-bf96-8b8f95cf0b42 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:27.231171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60055553-47ec-43d8-bf96-8b8f95cf0b42 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:27.232742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 686/1380] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:27.238027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba567f6a-16d3-4e38-82dc-d3f52d46adb4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:27.238572 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba567f6a-16d3-4e38-82dc-d3f52d46adb4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:27.238572 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba567f6a-16d3-4e38-82dc-d3f52d46adb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:27.249853 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:27.313198 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ba567f6a-16d3-4e38-82dc-d3f52d46adb4 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:27.314428 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ba567f6a-16d3-4e38-82dc-d3f52d46adb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:27.420247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 695/1381] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:27.465486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60055553-47ec-43d8-bf96-8b8f95cf0b42 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:02:27.467393 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-60055553-47ec-43d8-bf96-8b8f95cf0b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:27.628015 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:27.641811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60055553-47ec-43d8-bf96-8b8f95cf0b42 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:02:27.775809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e4681-9658-4abe-982f-a0acb042274b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:02:27.776602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e4681-9658-4abe-982f-a0acb042274b service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:27.776897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e4681-9658-4abe-982f-a0acb042274b service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:27.777684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e4681-9658-4abe-982f-a0acb042274b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:27.779659 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6c2e4681-9658-4abe-982f-a0acb042274b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:27.945535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e4681-9658-4abe-982f-a0acb042274b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:02:27.948699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e4681-9658-4abe-982f-a0acb042274b service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:02:27.969769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:27.988102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 687/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:02:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:02:30.254578 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f29e5-389c-4de2-bee8-0027d454e140 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:30.255200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f29e5-389c-4de2-bee8-0027d454e140 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:30.255200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f29e5-389c-4de2-bee8-0027d454e140 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:30.256930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 696/1383] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:30.262516 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc3750-d874-48fe-840f-1d72c64725f7 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:30.263183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc3750-d874-48fe-840f-1d72c64725f7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:30.263183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc3750-d874-48fe-840f-1d72c64725f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:30.330152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-acdc3750-d874-48fe-840f-1d72c64725f7 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:30.331374 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-acdc3750-d874-48fe-840f-1d72c64725f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:30.434360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 688/1384] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:30.476186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f29e5-389c-4de2-bee8-0027d454e140 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:02:30.477854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7b5f29e5-389c-4de2-bee8-0027d454e140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:30.574243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:30.643482 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f29e5-389c-4de2-bee8-0027d454e140 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:02:30.770597 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23134e1-f421-49d9-ad26-3eef98430dad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:02:30.771234 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23134e1-f421-49d9-ad26-3eef98430dad service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:30.771512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23134e1-f421-49d9-ad26-3eef98430dad service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:30.771780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23134e1-f421-49d9-ad26-3eef98430dad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:30.773455 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d23134e1-f421-49d9-ad26-3eef98430dad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:30.832630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:30.904309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23134e1-f421-49d9-ad26-3eef98430dad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:02:30.907961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23134e1-f421-49d9-ad26-3eef98430dad service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:02:30.932757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 697/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:02:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:02:33.013657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc39045-2fe0-4d22-b0c4-345c4c335f2c None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:33.014604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc39045-2fe0-4d22-b0c4-345c4c335f2c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:33.014604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc39045-2fe0-4d22-b0c4-345c4c335f2c None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:33.015503 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 689/1386] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:02:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:33.021559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea301d0-f69e-4c30-acb7-4372fff36345 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:33.022543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea301d0-f69e-4c30-acb7-4372fff36345 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:33.022543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea301d0-f69e-4c30-acb7-4372fff36345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:33.123675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-cea301d0-f69e-4c30-acb7-4372fff36345 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:33.124958 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cea301d0-f69e-4c30-acb7-4372fff36345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:33.228765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 698/1387] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:33.282605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc39045-2fe0-4d22-b0c4-345c4c335f2c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:02:33.284173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6bc39045-2fe0-4d22-b0c4-345c4c335f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:33.374699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:33.449533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc39045-2fe0-4d22-b0c4-345c4c335f2c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:02:33.566354 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcc140-e48b-4a0c-9018-9e4a16f666a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:02:33.566926 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcc140-e48b-4a0c-9018-9e4a16f666a6 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:33.567203 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcc140-e48b-4a0c-9018-9e4a16f666a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:33.567474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcc140-e48b-4a0c-9018-9e4a16f666a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:33.569137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-57bcc140-e48b-4a0c-9018-9e4a16f666a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:33.610338 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:33.666705 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcc140-e48b-4a0c-9018-9e4a16f666a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:02:33.669074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcc140-e48b-4a0c-9018-9e4a16f666a6 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:02:33.688723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 690/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:02:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:02:35.829319 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaeae71-b393-4d11-8757-2f37209aecfc None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:35.829886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaeae71-b393-4d11-8757-2f37209aecfc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:35.829886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaeae71-b393-4d11-8757-2f37209aecfc None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:35.831260 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 699/1389] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:35.836895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c370dd-5dd0-449a-876e-0c50053e59a4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:35.837371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c370dd-5dd0-449a-876e-0c50053e59a4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:35.837476 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c370dd-5dd0-449a-876e-0c50053e59a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:35.910822 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-31c370dd-5dd0-449a-876e-0c50053e59a4 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:35.912086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-31c370dd-5dd0-449a-876e-0c50053e59a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:36.016360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 691/1390] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:36.064705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaeae71-b393-4d11-8757-2f37209aecfc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:02:36.066269 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ddaeae71-b393-4d11-8757-2f37209aecfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:36.099463 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:36.223212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaeae71-b393-4d11-8757-2f37209aecfc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:02:36.345725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:36.360238 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5eecb4-d1a6-4f3b-9e7b-0e102e411b1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:02:36.361072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5eecb4-d1a6-4f3b-9e7b-0e102e411b1c service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:36.361358 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5eecb4-d1a6-4f3b-9e7b-0e102e411b1c service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:36.361645 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5eecb4-d1a6-4f3b-9e7b-0e102e411b1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:36.363412 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ca5eecb4-d1a6-4f3b-9e7b-0e102e411b1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:36.478156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5eecb4-d1a6-4f3b-9e7b-0e102e411b1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:02:36.479407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5eecb4-d1a6-4f3b-9e7b-0e102e411b1c service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:02:36.502979 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 700/1391] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:02:37.077577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f7c08-2d64-4266-83d6-b3b1ec98a66f None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:37.078375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f7c08-2d64-4266-83d6-b3b1ec98a66f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:37.078375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f7c08-2d64-4266-83d6-b3b1ec98a66f None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:37.079621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 692/1392] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:37.086885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5f3ad-0bcf-4a63-87c5-deef7d757ac4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:37.087442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5f3ad-0bcf-4a63-87c5-deef7d757ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:37.087442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5f3ad-0bcf-4a63-87c5-deef7d757ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:37.173987 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-02c5f3ad-0bcf-4a63-87c5-deef7d757ac4 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:37.175253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-02c5f3ad-0bcf-4a63-87c5-deef7d757ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:37.284693 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:37.338570 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 701/1393] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:37.405370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb5895-d32b-480f-8965-16785f73dae3 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:37.405950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb5895-d32b-480f-8965-16785f73dae3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:37.406488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb5895-d32b-480f-8965-16785f73dae3 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:37.408650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 693/1394] 10.222.0.13 () {54 vars in 966 bytes} [Thu Jul 21 09:02:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:37.416833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fd155-8630-491b-bef8-dc9cf2ef8816 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:37.417122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fd155-8630-491b-bef8-dc9cf2ef8816 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:37.417395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29fd155-8630-491b-bef8-dc9cf2ef8816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:37.521101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d29fd155-8630-491b-bef8-dc9cf2ef8816 None None] MFA Rules not processed for user `e053b6b25a4e4a20a05e6ffa6683222b`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:37.522368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d29fd155-8630-491b-bef8-dc9cf2ef8816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:37.636370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 702/1395] 10.222.0.13 () {58 vars in 1063 bytes} [Thu Jul 21 09:02:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:37.638428 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb5895-d32b-480f-8965-16785f73dae3 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:02:37.639952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-05bb5895-d32b-480f-8965-16785f73dae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:37.654384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:37.774951 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb5895-d32b-480f-8965-16785f73dae3 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:02:37.854151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6d765-7b4d-4540-a6c3-ac8122caeee8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e053b6b25a4e4a20a05e6ffa6683222b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:02:37.855436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6d765-7b4d-4540-a6c3-ac8122caeee8 service glance] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:37.855723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6d765-7b4d-4540-a6c3-ac8122caeee8 service glance] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:37.856031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6d765-7b4d-4540-a6c3-ac8122caeee8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:37.857675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7ab6d765-7b4d-4540-a6c3-ac8122caeee8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:37.942432 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:37.990193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6d765-7b4d-4540-a6c3-ac8122caeee8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:02:37.991709 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6d765-7b4d-4540-a6c3-ac8122caeee8 service glance] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:02:38.013725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 694/1396] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:02:38.654430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f9457-ddf2-4de7-a16e-f7955cf255ae None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:38.655054 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f9457-ddf2-4de7-a16e-f7955cf255ae None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:38.655054 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f9457-ddf2-4de7-a16e-f7955cf255ae None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:38.656455 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 703/1397] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:38.662473 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f54f2-7849-41e1-812e-6e4ef952964d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:38.663059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f54f2-7849-41e1-812e-6e4ef952964d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:38.663263 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f54f2-7849-41e1-812e-6e4ef952964d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:38.738846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-372f54f2-7849-41e1-812e-6e4ef952964d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:38.740117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-372f54f2-7849-41e1-812e-6e4ef952964d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:38.760020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18505c67-cc4b-4081-b3bd-0e2b1dfbb882 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:38.760310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18505c67-cc4b-4081-b3bd-0e2b1dfbb882 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:38.760615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18505c67-cc4b-4081-b3bd-0e2b1dfbb882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:38.830302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-18505c67-cc4b-4081-b3bd-0e2b1dfbb882 None None] MFA Rules not processed for user `257450db238d4efda783e30cbcd23d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:38.831569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-18505c67-cc4b-4081-b3bd-0e2b1dfbb882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:38.840452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:38.842185 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 695/1398] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:38.887285 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f54f2-7849-41e1-812e-6e4ef952964d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:02:38.889088 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-372f54f2-7849-41e1-812e-6e4ef952964d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:38.916637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:38.925354 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 704/1399] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jul 21 09:02:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:02:38.974772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156eaaae-b912-459c-999a-dfc9b7314dc0 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:38.975108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156eaaae-b912-459c-999a-dfc9b7314dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:38.975399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156eaaae-b912-459c-999a-dfc9b7314dc0 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:38.976979 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 705/1400] 10.222.0.13 () {54 vars in 943 bytes} [Thu Jul 21 09:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 09:02:38.983947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24595e47-28c0-447b-a798-c0723c314bf4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:38.984245 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24595e47-28c0-447b-a798-c0723c314bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:38.984534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24595e47-28c0-447b-a798-c0723c314bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:39.015096 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f54f2-7849-41e1-812e-6e4ef952964d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:02:39.063769 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-24595e47-28c0-447b-a798-c0723c314bf4 None None] MFA Rules not processed for user `061def74ec7f42ce9a174e0944bf499f`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:39.065038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-24595e47-28c0-447b-a798-c0723c314bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:39.139137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bd4e4-b7c1-4998-b8f8-9826fba40c18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:02:39.139792 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bd4e4-b7c1-4998-b8f8-9826fba40c18 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:39.140113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bd4e4-b7c1-4998-b8f8-9826fba40c18 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:39.140391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bd4e4-b7c1-4998-b8f8-9826fba40c18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:39.142061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8d5bd4e4-b7c1-4998-b8f8-9826fba40c18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:39.159427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:39.162247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:39.183425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 706/1401] 10.222.0.13 () {58 vars in 1040 bytes} [Thu Jul 21 09:02:38 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:02:39.192678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24595e47-28c0-447b-a798-c0723c314bf4 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:02:39.202278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24595e47-28c0-447b-a798-c0723c314bf4 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:02:39.241233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bd4e4-b7c1-4998-b8f8-9826fba40c18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:02:39.242866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bd4e4-b7c1-4998-b8f8-9826fba40c18 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:02:39.261901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 696/1402] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:02:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:02:39.311797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09cb4a6-3aec-4cf7-9637-2db634a6873e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061def74ec7f42ce9a174e0944bf499f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:02:39.312482 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09cb4a6-3aec-4cf7-9637-2db634a6873e service swift] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:39.312771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09cb4a6-3aec-4cf7-9637-2db634a6873e service swift] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:39.313075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09cb4a6-3aec-4cf7-9637-2db634a6873e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:39.315139 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c09cb4a6-3aec-4cf7-9637-2db634a6873e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:39.410442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09cb4a6-3aec-4cf7-9637-2db634a6873e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:02:39.412102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09cb4a6-3aec-4cf7-9637-2db634a6873e service swift] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:02:39.421747 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 707/1403] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jul 21 09:02:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 09:02:41.408587 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2e7ba-4e96-4999-a793-9e8d6cd35a19 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:41.409191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2e7ba-4e96-4999-a793-9e8d6cd35a19 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:41.409191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2e7ba-4e96-4999-a793-9e8d6cd35a19 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:41.412307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 697/1404] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:02:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:41.416297 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83205a5-4965-4932-a4a8-3f89b00dc592 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:41.416773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83205a5-4965-4932-a4a8-3f89b00dc592 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:41.416864 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83205a5-4965-4932-a4a8-3f89b00dc592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:41.428727 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:41.493856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b83205a5-4965-4932-a4a8-3f89b00dc592 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:41.495427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b83205a5-4965-4932-a4a8-3f89b00dc592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:41.619306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 708/1405] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:41.666086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2e7ba-4e96-4999-a793-9e8d6cd35a19 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:02:41.668318 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-38b2e7ba-4e96-4999-a793-9e8d6cd35a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:41.847089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:41.850535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2e7ba-4e96-4999-a793-9e8d6cd35a19 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:02:42.001595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada796a-66a4-4815-be8b-4063eb104727 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:02:42.002177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada796a-66a4-4815-be8b-4063eb104727 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:42.002672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada796a-66a4-4815-be8b-4063eb104727 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:42.002949 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada796a-66a4-4815-be8b-4063eb104727 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:42.004776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-eada796a-66a4-4815-be8b-4063eb104727 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:42.113920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada796a-66a4-4815-be8b-4063eb104727 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:02:42.115543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada796a-66a4-4815-be8b-4063eb104727 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:02:42.140463 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 698/1406] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:02:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:02:44.310634 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31fc7a-3764-440c-ae92-2f2b00da619f None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:44.311274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31fc7a-3764-440c-ae92-2f2b00da619f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:44.311274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31fc7a-3764-440c-ae92-2f2b00da619f None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:44.312573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 709/1407] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:02:44.317877 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7b76d-871b-4244-bcff-b376ded7cb0e None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:44.318352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7b76d-871b-4244-bcff-b376ded7cb0e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:44.318454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7b76d-871b-4244-bcff-b376ded7cb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:44.321967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:44.396736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5be7b76d-871b-4244-bcff-b376ded7cb0e None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:02:44.397963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5be7b76d-871b-4244-bcff-b376ded7cb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:44.516261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 699/1408] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:02:44.563144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31fc7a-3764-440c-ae92-2f2b00da619f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:02:44.565052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7b31fc7a-3764-440c-ae92-2f2b00da619f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:44.675780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:44.751158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31fc7a-3764-440c-ae92-2f2b00da619f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:02:44.892307 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de239d-7d2a-4d5a-9db9-f0dbec901776 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:02:44.892896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de239d-7d2a-4d5a-9db9-f0dbec901776 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:02:44.893400 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de239d-7d2a-4d5a-9db9-f0dbec901776 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:02:44.893549 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de239d-7d2a-4d5a-9db9-f0dbec901776 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:02:44.895273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c9de239d-7d2a-4d5a-9db9-f0dbec901776 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:02:44.958058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:45.006250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de239d-7d2a-4d5a-9db9-f0dbec901776 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:02:45.008135 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de239d-7d2a-4d5a-9db9-f0dbec901776 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:02:45.032305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 710/1409] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:02:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:02:54.513773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=64 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:02:55.028726 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=23 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:02.332692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f81f5-71ef-489d-9f24-7d25105d7218 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:02.333268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f81f5-71ef-489d-9f24-7d25105d7218 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:02.333268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f81f5-71ef-489d-9f24-7d25105d7218 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:02.334625 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 700/1410] 10.222.0.13 () {54 vars in 918 bytes} [Thu Jul 21 09:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:02.340942 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161d910-d63e-42f3-b9bf-aa0e6a1442bf None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:02.340942 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161d910-d63e-42f3-b9bf-aa0e6a1442bf None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:02.340942 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161d910-d63e-42f3-b9bf-aa0e6a1442bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:02.416553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e161d910-d63e-42f3-b9bf-aa0e6a1442bf None None] MFA Rules not processed for user `72f9ef6de66e456e800237e9f954e98d`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:02.418375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e161d910-d63e-42f3-b9bf-aa0e6a1442bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:02.533076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 711/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jul 21 09:03:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:03:02.547394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddea67f-2411-46f1-96e6-68758b384579 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:02.547681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddea67f-2411-46f1-96e6-68758b384579 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:02.547991 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddea67f-2411-46f1-96e6-68758b384579 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:02.549396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 701/1412] 10.222.0.13 () {54 vars in 918 bytes} [Thu Jul 21 09:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:02.554514 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ea610-a523-46b5-b38e-cdc234a650ac None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:02.554798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ea610-a523-46b5-b38e-cdc234a650ac None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:02.555073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ea610-a523-46b5-b38e-cdc234a650ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:02.631153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:02.634703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a62ea610-a523-46b5-b38e-cdc234a650ac None None] MFA Rules not processed for user `c48d41dcf9b344a8b2fec77f4dd75fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:02.636796 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a62ea610-a523-46b5-b38e-cdc234a650ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:02.759567 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 712/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jul 21 09:03:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:05.297327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dce31-fe93-40f1-8ed6-bc60fd8efd82 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:05.297870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dce31-fe93-40f1-8ed6-bc60fd8efd82 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:05.297870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dce31-fe93-40f1-8ed6-bc60fd8efd82 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:05.299297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 09:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:05.304991 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b49abc-2548-466e-90fe-d9abce976e15 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:05.305400 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b49abc-2548-466e-90fe-d9abce976e15 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:05.305557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b49abc-2548-466e-90fe-d9abce976e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:05.372181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c6b49abc-2548-466e-90fe-d9abce976e15 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:05.373399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c6b49abc-2548-466e-90fe-d9abce976e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:05.429176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:05.479550 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 09:03:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:05.525062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dce31-fe93-40f1-8ed6-bc60fd8efd82 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:05.526611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e14dce31-fe93-40f1-8ed6-bc60fd8efd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:05.676658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dce31-fe93-40f1-8ed6-bc60fd8efd82 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:05.752372 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:05.795712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3af2ce-9375-4c4d-adac-2917fac6aab4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:05.796396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3af2ce-9375-4c4d-adac-2917fac6aab4 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:05.796687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3af2ce-9375-4c4d-adac-2917fac6aab4 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:05.796963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3af2ce-9375-4c4d-adac-2917fac6aab4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:05.798627 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0f3af2ce-9375-4c4d-adac-2917fac6aab4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:05.897023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3af2ce-9375-4c4d-adac-2917fac6aab4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:05.898593 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3af2ce-9375-4c4d-adac-2917fac6aab4 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:05.922986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 703/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:06.842530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e6767-938b-4bb2-981b-ce52aef94e2e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:06.843092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e6767-938b-4bb2-981b-ce52aef94e2e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:06.843196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e6767-938b-4bb2-981b-ce52aef94e2e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:06.844569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 09:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:06.849610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d79e7-4751-4d86-9142-645654130ae3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:06.850021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d79e7-4751-4d86-9142-645654130ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:06.850173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d79e7-4751-4d86-9142-645654130ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:06.928402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-234d79e7-4751-4d86-9142-645654130ae3 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:06.929625 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-234d79e7-4751-4d86-9142-645654130ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:06.958321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:07.030182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 09:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:07.181915 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e6767-938b-4bb2-981b-ce52aef94e2e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:07.184785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6a6e6767-938b-4bb2-981b-ce52aef94e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:07.341777 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e6767-938b-4bb2-981b-ce52aef94e2e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:07.382652 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:07.471764 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331af48-3ba0-42de-813d-f490cd9c0486 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:07.472457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331af48-3ba0-42de-813d-f490cd9c0486 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:07.472742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331af48-3ba0-42de-813d-f490cd9c0486 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:07.473018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331af48-3ba0-42de-813d-f490cd9c0486 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:07.474730 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1331af48-3ba0-42de-813d-f490cd9c0486 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:07.567355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331af48-3ba0-42de-813d-f490cd9c0486 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:07.568993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331af48-3ba0-42de-813d-f490cd9c0486 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:07.587172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 715/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:08.506106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a935e-f4f4-4b96-b460-f9fd42506c14 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:08.506659 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a935e-f4f4-4b96-b460-f9fd42506c14 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:08.506659 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a935e-f4f4-4b96-b460-f9fd42506c14 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:08.508049 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 09:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 09:03:08.513501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0487318-7214-4d51-b967-771ab574111e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:08.513935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0487318-7214-4d51-b967-771ab574111e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:08.514064 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0487318-7214-4d51-b967-771ab574111e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:08.577272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:08.599571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d0487318-7214-4d51-b967-771ab574111e None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:08.600829 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d0487318-7214-4d51-b967-771ab574111e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:08.706422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 09:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:08.859391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a935e-f4f4-4b96-b460-f9fd42506c14 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:08.860878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fe2a935e-f4f4-4b96-b460-f9fd42506c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:09.009115 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a935e-f4f4-4b96-b460-f9fd42506c14 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:09.021600 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:09.128780 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e9ce1-6aee-4758-a6dc-57a327af61cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:09.129344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e9ce1-6aee-4758-a6dc-57a327af61cf service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:09.129625 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e9ce1-6aee-4758-a6dc-57a327af61cf service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:09.129899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e9ce1-6aee-4758-a6dc-57a327af61cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:09.131543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-177e9ce1-6aee-4758-a6dc-57a327af61cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:09.224190 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e9ce1-6aee-4758-a6dc-57a327af61cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:09.225814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e9ce1-6aee-4758-a6dc-57a327af61cf service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:09.244351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 706/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:10.122528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b2553-d99a-43b5-8cf5-c4b20f063a03 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:10.123080 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b2553-d99a-43b5-8cf5-c4b20f063a03 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:10.123080 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b2553-d99a-43b5-8cf5-c4b20f063a03 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:10.124663 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 09:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:10.129897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15182a02-3fb7-4e27-9887-4fdcc04889d3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:10.130360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15182a02-3fb7-4e27-9887-4fdcc04889d3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:10.130489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15182a02-3fb7-4e27-9887-4fdcc04889d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:10.149735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:10.208017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-15182a02-3fb7-4e27-9887-4fdcc04889d3 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:10.209242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-15182a02-3fb7-4e27-9887-4fdcc04889d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:10.305911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 09:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:10.460203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b2553-d99a-43b5-8cf5-c4b20f063a03 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:10.461749 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ec1b2553-d99a-43b5-8cf5-c4b20f063a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:10.572757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:10.609865 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b2553-d99a-43b5-8cf5-c4b20f063a03 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:10.724654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a203c-a489-4aeb-8ce1-14c98fabb724 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:10.725343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a203c-a489-4aeb-8ce1-14c98fabb724 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:10.725582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a203c-a489-4aeb-8ce1-14c98fabb724 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:10.725854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a203c-a489-4aeb-8ce1-14c98fabb724 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:10.727493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-236a203c-a489-4aeb-8ce1-14c98fabb724 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:10.805261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:10.819680 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a203c-a489-4aeb-8ce1-14c98fabb724 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:10.821342 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a203c-a489-4aeb-8ce1-14c98fabb724 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:10.840304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 718/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:11.722854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a80cb-300c-4259-822c-deb6abb78e13 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:11.723330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a80cb-300c-4259-822c-deb6abb78e13 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:11.723447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a80cb-300c-4259-822c-deb6abb78e13 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:11.724834 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 09:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:11.730841 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea7350-ad60-4aa9-8b61-03041d6bdff1 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:11.731324 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea7350-ad60-4aa9-8b61-03041d6bdff1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:11.731420 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea7350-ad60-4aa9-8b61-03041d6bdff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:11.808949 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-06ea7350-ad60-4aa9-8b61-03041d6bdff1 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:11.810204 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-06ea7350-ad60-4aa9-8b61-03041d6bdff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:11.913886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 09:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:12.067079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a80cb-300c-4259-822c-deb6abb78e13 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:12.068658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-483a80cb-300c-4259-822c-deb6abb78e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:12.156988 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:12.222489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a80cb-300c-4259-822c-deb6abb78e13 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:12.342060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026dff8-aefe-42ee-b6c2-366cee1bcc6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:12.342735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026dff8-aefe-42ee-b6c2-366cee1bcc6d service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:12.343018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026dff8-aefe-42ee-b6c2-366cee1bcc6d service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:12.343293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026dff8-aefe-42ee-b6c2-366cee1bcc6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:12.344971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2026dff8-aefe-42ee-b6c2-366cee1bcc6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:12.394438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:12.438744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026dff8-aefe-42ee-b6c2-366cee1bcc6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:12.440457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026dff8-aefe-42ee-b6c2-366cee1bcc6d service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:12.460383 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 709/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:13.286277 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8b7b1-5a86-4b5a-8c9d-0ec129e115a3 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:13.286935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8b7b1-5a86-4b5a-8c9d-0ec129e115a3 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:13.286935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8b7b1-5a86-4b5a-8c9d-0ec129e115a3 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:13.288311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 09:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:13.293970 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249b8fe-3549-402a-a402-b546ca0543b7 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:13.294460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249b8fe-3549-402a-a402-b546ca0543b7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:13.294563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249b8fe-3549-402a-a402-b546ca0543b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:13.373952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-7249b8fe-3549-402a-a402-b546ca0543b7 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:13.375226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7249b8fe-3549-402a-a402-b546ca0543b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:13.475320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 09:03:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:13.627290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8b7b1-5a86-4b5a-8c9d-0ec129e115a3 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:13.628871 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fad8b7b1-5a86-4b5a-8c9d-0ec129e115a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:13.681260 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:13.783377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8b7b1-5a86-4b5a-8c9d-0ec129e115a3 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:13.903724 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3b476-3e7c-471b-8529-1962890e41c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:13.904404 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3b476-3e7c-471b-8529-1962890e41c3 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:13.904686 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3b476-3e7c-471b-8529-1962890e41c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:13.904959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3b476-3e7c-471b-8529-1962890e41c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:13.906610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fab3b476-3e7c-471b-8529-1962890e41c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:13.913851 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:14.001506 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3b476-3e7c-471b-8529-1962890e41c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:14.003183 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3b476-3e7c-471b-8529-1962890e41c3 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:14.021584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 721/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:14.860579 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1dd18-6d65-4ddd-910a-71d86bd46c39 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:14.861118 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1dd18-6d65-4ddd-910a-71d86bd46c39 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:14.861118 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1dd18-6d65-4ddd-910a-71d86bd46c39 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:14.862494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 09:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:14.867997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be68a3f-986d-44b6-bf7c-08686b38924e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:14.868556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be68a3f-986d-44b6-bf7c-08686b38924e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:14.868556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be68a3f-986d-44b6-bf7c-08686b38924e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:14.946948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7be68a3f-986d-44b6-bf7c-08686b38924e None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:14.948357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7be68a3f-986d-44b6-bf7c-08686b38924e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:15.018329 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:15.049647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 09:03:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:15.202185 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1dd18-6d65-4ddd-910a-71d86bd46c39 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:15.203824 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1ce1dd18-6d65-4ddd-910a-71d86bd46c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:15.222103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:15.353847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1dd18-6d65-4ddd-910a-71d86bd46c39 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:15.450302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:15.478113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18cd30-d457-457f-b793-f749a505ee22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:15.478694 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18cd30-d457-457f-b793-f749a505ee22 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:15.479496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18cd30-d457-457f-b793-f749a505ee22 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:15.479496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18cd30-d457-457f-b793-f749a505ee22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:15.481126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2c18cd30-d457-457f-b793-f749a505ee22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:15.577448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18cd30-d457-457f-b793-f749a505ee22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:15.579117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18cd30-d457-457f-b793-f749a505ee22 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:15.598018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 712/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:16.449662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb288653-6f8c-4e74-a9c8-9cb23e36d673 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:16.450246 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb288653-6f8c-4e74-a9c8-9cb23e36d673 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:16.450386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb288653-6f8c-4e74-a9c8-9cb23e36d673 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:16.452003 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 09:03:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:16.457571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9390a31-8558-4e65-8d62-f8953dd01c69 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:16.458093 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9390a31-8558-4e65-8d62-f8953dd01c69 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:16.458093 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9390a31-8558-4e65-8d62-f8953dd01c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:16.544559 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b9390a31-8558-4e65-8d62-f8953dd01c69 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:16.545803 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b9390a31-8558-4e65-8d62-f8953dd01c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:16.591456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:16.658009 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 09:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:16.808072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb288653-6f8c-4e74-a9c8-9cb23e36d673 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:16.809661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-eb288653-6f8c-4e74-a9c8-9cb23e36d673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:16.961320 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb288653-6f8c-4e74-a9c8-9cb23e36d673 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:17.016631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:17.081691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec876b48-a877-4d1a-91de-fcd034a24c6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:17.082343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec876b48-a877-4d1a-91de-fcd034a24c6f service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:17.082623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec876b48-a877-4d1a-91de-fcd034a24c6f service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:17.082894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec876b48-a877-4d1a-91de-fcd034a24c6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:17.085453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ec876b48-a877-4d1a-91de-fcd034a24c6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:17.183577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec876b48-a877-4d1a-91de-fcd034a24c6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:17.185127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec876b48-a877-4d1a-91de-fcd034a24c6f service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:17.203944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 724/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:18.086939 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5014a-28eb-40fd-be86-081d19311415 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:18.087485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5014a-28eb-40fd-be86-081d19311415 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:18.087485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5014a-28eb-40fd-be86-081d19311415 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:18.089390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 09:03:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:18.094843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dcf92-214e-4dd8-be4e-e9f6cf3c9d80 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:18.095430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dcf92-214e-4dd8-be4e-e9f6cf3c9d80 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:18.095430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dcf92-214e-4dd8-be4e-e9f6cf3c9d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:18.173286 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-330dcf92-214e-4dd8-be4e-e9f6cf3c9d80 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:18.174503 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-330dcf92-214e-4dd8-be4e-e9f6cf3c9d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:18.176649 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:18.274956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 09:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:18.425831 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5014a-28eb-40fd-be86-081d19311415 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:18.427488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c6b5014a-28eb-40fd-be86-081d19311415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:18.576889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5014a-28eb-40fd-be86-081d19311415 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:18.604779 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:18.695584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910e84b-11bd-4363-995d-cf04bfa589ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:18.696171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910e84b-11bd-4363-995d-cf04bfa589ea service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:18.696456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910e84b-11bd-4363-995d-cf04bfa589ea service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:18.696728 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910e84b-11bd-4363-995d-cf04bfa589ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:18.698434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a910e84b-11bd-4363-995d-cf04bfa589ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:18.797079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910e84b-11bd-4363-995d-cf04bfa589ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:18.798712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910e84b-11bd-4363-995d-cf04bfa589ea service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:18.817215 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 715/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:20.766109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07969-fa5f-423d-9eda-67f26d594884 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:20.766635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07969-fa5f-423d-9eda-67f26d594884 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:20.766635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07969-fa5f-423d-9eda-67f26d594884 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:20.768048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 09:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:20.773294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96027f-a03d-44e6-a3da-cf05700ba9d3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:20.773712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96027f-a03d-44e6-a3da-cf05700ba9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:20.773846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96027f-a03d-44e6-a3da-cf05700ba9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:20.813332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:20.839307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2c96027f-a03d-44e6-a3da-cf05700ba9d3 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:20.840532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2c96027f-a03d-44e6-a3da-cf05700ba9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:20.946096 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 09:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:20.986128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e07969-fa5f-423d-9eda-67f26d594884 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:20.987664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-55e07969-fa5f-423d-9eda-67f26d594884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:21.091502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e07969-fa5f-423d-9eda-67f26d594884 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:21.124251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:21.166196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9502c6-05ba-4608-a507-cac60442ef04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:21.166850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9502c6-05ba-4608-a507-cac60442ef04 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:21.167126 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9502c6-05ba-4608-a507-cac60442ef04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:21.167400 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9502c6-05ba-4608-a507-cac60442ef04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:21.169086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7d9502c6-05ba-4608-a507-cac60442ef04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:21.279263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9502c6-05ba-4608-a507-cac60442ef04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:21.280800 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9502c6-05ba-4608-a507-cac60442ef04 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:21.299224 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 727/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:23.584736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e013867-f2fb-4ab8-b776-45113ca57dfe None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:23.585303 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e013867-f2fb-4ab8-b776-45113ca57dfe None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:23.585303 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e013867-f2fb-4ab8-b776-45113ca57dfe None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:23.586694 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 09:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:23.592294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89771f29-4162-4ba4-a9a2-0578b9f0c50f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:23.592751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89771f29-4162-4ba4-a9a2-0578b9f0c50f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:23.592848 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89771f29-4162-4ba4-a9a2-0578b9f0c50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:23.661713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-89771f29-4162-4ba4-a9a2-0578b9f0c50f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:23.662921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-89771f29-4162-4ba4-a9a2-0578b9f0c50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:23.676368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:23.764131 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 09:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:23.802735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e013867-f2fb-4ab8-b776-45113ca57dfe None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:23.804550 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1e013867-f2fb-4ab8-b776-45113ca57dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:23.909942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e013867-f2fb-4ab8-b776-45113ca57dfe None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:23.922670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:23.990504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df0bd5-117a-449e-995b-a92bd1397aa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:23.991158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df0bd5-117a-449e-995b-a92bd1397aa4 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:23.991446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df0bd5-117a-449e-995b-a92bd1397aa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:23.991727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df0bd5-117a-449e-995b-a92bd1397aa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:23.993418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-65df0bd5-117a-449e-995b-a92bd1397aa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:24.109494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df0bd5-117a-449e-995b-a92bd1397aa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:24.111577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df0bd5-117a-449e-995b-a92bd1397aa4 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:24.135390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 718/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:26.484228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eba617-7129-4a71-bee1-b7dc2e39c310 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:26.484874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eba617-7129-4a71-bee1-b7dc2e39c310 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:26.484874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eba617-7129-4a71-bee1-b7dc2e39c310 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:26.486189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 09:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:26.491402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa773e-b649-47eb-a3c0-af66510a45cf None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:26.491801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa773e-b649-47eb-a3c0-af66510a45cf None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:26.491999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa773e-b649-47eb-a3c0-af66510a45cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:26.540010 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:26.558868 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e8aa773e-b649-47eb-a3c0-af66510a45cf None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:26.560103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e8aa773e-b649-47eb-a3c0-af66510a45cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:26.656129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 09:03:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:26.695728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eba617-7129-4a71-bee1-b7dc2e39c310 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:26.697913 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b2eba617-7129-4a71-bee1-b7dc2e39c310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:26.805901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eba617-7129-4a71-bee1-b7dc2e39c310 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:26.845915 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:26.879633 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b04948-36f7-4d3e-a4c5-c8e93f13652b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:26.880318 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b04948-36f7-4d3e-a4c5-c8e93f13652b service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:26.880601 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b04948-36f7-4d3e-a4c5-c8e93f13652b service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:26.880879 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b04948-36f7-4d3e-a4c5-c8e93f13652b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:26.882517 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f1b04948-36f7-4d3e-a4c5-c8e93f13652b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:26.996975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b04948-36f7-4d3e-a4c5-c8e93f13652b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:26.998427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b04948-36f7-4d3e-a4c5-c8e93f13652b service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:27.017618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 730/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:29.195933 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1e359-6da5-46d2-bfad-c8cafceda31a None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:29.196480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1e359-6da5-46d2-bfad-c8cafceda31a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:29.196480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1e359-6da5-46d2-bfad-c8cafceda31a None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:29.199052 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 09:03:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:29.204272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae0374-5544-4e7d-82da-86ed38a35f57 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:29.204758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae0374-5544-4e7d-82da-86ed38a35f57 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:29.204888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae0374-5544-4e7d-82da-86ed38a35f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:29.271293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-acae0374-5544-4e7d-82da-86ed38a35f57 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:29.272536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-acae0374-5544-4e7d-82da-86ed38a35f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:29.294102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:29.370786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 09:03:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:29.410757 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1e359-6da5-46d2-bfad-c8cafceda31a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:29.412409 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c5f1e359-6da5-46d2-bfad-c8cafceda31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:29.528498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1e359-6da5-46d2-bfad-c8cafceda31a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:29.548571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:29.602936 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35206b3-3f90-4dea-96fd-de35d845e4ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:29.604383 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35206b3-3f90-4dea-96fd-de35d845e4ce service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:29.604570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35206b3-3f90-4dea-96fd-de35d845e4ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:29.604770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35206b3-3f90-4dea-96fd-de35d845e4ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:29.606454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f35206b3-3f90-4dea-96fd-de35d845e4ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:29.718430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35206b3-3f90-4dea-96fd-de35d845e4ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:29.719906 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35206b3-3f90-4dea-96fd-de35d845e4ce service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:29.739003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 721/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:31.857202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905f181-6040-4ebb-80dd-e8bb098001ae None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:31.857787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905f181-6040-4ebb-80dd-e8bb098001ae None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:31.857787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905f181-6040-4ebb-80dd-e8bb098001ae None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:31.859120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 09:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:31.864490 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc852c6-ad06-4dc9-b624-1a086f203ec3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:31.864951 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc852c6-ad06-4dc9-b624-1a086f203ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:31.865070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc852c6-ad06-4dc9-b624-1a086f203ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:31.933446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:31.936625 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-cdc852c6-ad06-4dc9-b624-1a086f203ec3 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:31.938291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-cdc852c6-ad06-4dc9-b624-1a086f203ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:32.036956 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 09:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:32.076935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905f181-6040-4ebb-80dd-e8bb098001ae None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:32.078492 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7905f181-6040-4ebb-80dd-e8bb098001ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:32.188944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905f181-6040-4ebb-80dd-e8bb098001ae None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:32.240769 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:32.266764 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ceae4-842e-4b72-9ac9-046e8a04c317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:32.267408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ceae4-842e-4b72-9ac9-046e8a04c317 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:32.267710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ceae4-842e-4b72-9ac9-046e8a04c317 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:32.267995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ceae4-842e-4b72-9ac9-046e8a04c317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:32.269645 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fe2ceae4-842e-4b72-9ac9-046e8a04c317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:32.379574 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ceae4-842e-4b72-9ac9-046e8a04c317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:32.381052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ceae4-842e-4b72-9ac9-046e8a04c317 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:32.399930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 733/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:34.599021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55234425-7792-4664-a8d9-972c922d6ddb None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:34.599554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55234425-7792-4664-a8d9-972c922d6ddb None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:34.599554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55234425-7792-4664-a8d9-972c922d6ddb None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:34.600941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 09:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:34.607003 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61f1eb-39e8-45c0-8987-7a70b2a4a5ce None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:34.607474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61f1eb-39e8-45c0-8987-7a70b2a4a5ce None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:34.607568 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61f1eb-39e8-45c0-8987-7a70b2a4a5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:34.674334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7b61f1eb-39e8-45c0-8987-7a70b2a4a5ce None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:34.675552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7b61f1eb-39e8-45c0-8987-7a70b2a4a5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:34.705923 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:34.781147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 09:03:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:34.820803 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55234425-7792-4664-a8d9-972c922d6ddb None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:34.823593 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-55234425-7792-4664-a8d9-972c922d6ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:34.934176 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55234425-7792-4664-a8d9-972c922d6ddb None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:34.964154 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:35.011462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc6ef4-c5b3-4aa7-b193-054f418aa5f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:35.012155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc6ef4-c5b3-4aa7-b193-054f418aa5f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:35.012430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc6ef4-c5b3-4aa7-b193-054f418aa5f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:35.012703 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc6ef4-c5b3-4aa7-b193-054f418aa5f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:35.014348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c0bc6ef4-c5b3-4aa7-b193-054f418aa5f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:35.133949 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc6ef4-c5b3-4aa7-b193-054f418aa5f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:35.136146 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc6ef4-c5b3-4aa7-b193-054f418aa5f4 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:35.154896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 724/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:37.281537 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed224a8-f05c-4eb4-bfe5-f6a3e5f0fb02 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:37.282207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed224a8-f05c-4eb4-bfe5-f6a3e5f0fb02 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:37.282207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed224a8-f05c-4eb4-bfe5-f6a3e5f0fb02 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:37.283613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 09:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:37.289308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ec6e4-b372-4353-b574-4c29eaac2e39 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:37.289759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ec6e4-b372-4353-b574-4c29eaac2e39 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:37.289870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ec6e4-b372-4353-b574-4c29eaac2e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:37.357059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4e6ec6e4-b372-4353-b574-4c29eaac2e39 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:37.358292 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4e6ec6e4-b372-4353-b574-4c29eaac2e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:37.367712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:37.462062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 09:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:03:37.502642 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed224a8-f05c-4eb4-bfe5-f6a3e5f0fb02 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:37.504335 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5ed224a8-f05c-4eb4-bfe5-f6a3e5f0fb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:37.625713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed224a8-f05c-4eb4-bfe5-f6a3e5f0fb02 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:37.690952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:37.722250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b7afc-d761-44c6-bdd6-87dce0740ce5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:37.723127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b7afc-d761-44c6-bdd6-87dce0740ce5 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:37.723560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b7afc-d761-44c6-bdd6-87dce0740ce5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:37.723895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b7afc-d761-44c6-bdd6-87dce0740ce5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:37.725982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-876b7afc-d761-44c6-bdd6-87dce0740ce5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:37.862973 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b7afc-d761-44c6-bdd6-87dce0740ce5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:37.864457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b7afc-d761-44c6-bdd6-87dce0740ce5 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:37.883656 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 736/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:40.030070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f0d23-ee27-49e9-b914-8a09ffde7d30 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:40.030629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f0d23-ee27-49e9-b914-8a09ffde7d30 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:40.030629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f0d23-ee27-49e9-b914-8a09ffde7d30 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:40.031903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 09:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:40.037767 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5a5e8-6671-4296-bb0c-5f77cad1ed53 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:40.038250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5a5e8-6671-4296-bb0c-5f77cad1ed53 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:40.038359 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5a5e8-6671-4296-bb0c-5f77cad1ed53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:40.107345 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4ad5a5e8-6671-4296-bb0c-5f77cad1ed53 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:40.108746 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4ad5a5e8-6671-4296-bb0c-5f77cad1ed53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:40.153486 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:40.217157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 09:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:03:40.254940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f0d23-ee27-49e9-b914-8a09ffde7d30 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:40.256512 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bb3f0d23-ee27-49e9-b914-8a09ffde7d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:40.365992 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f0d23-ee27-49e9-b914-8a09ffde7d30 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:40.406550 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:40.453259 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e531f26-8009-4c41-996b-fa5de29bb202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:40.454010 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e531f26-8009-4c41-996b-fa5de29bb202 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:40.454354 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e531f26-8009-4c41-996b-fa5de29bb202 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:40.454709 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e531f26-8009-4c41-996b-fa5de29bb202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:40.456627 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9e531f26-8009-4c41-996b-fa5de29bb202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:40.578983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e531f26-8009-4c41-996b-fa5de29bb202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:40.580564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e531f26-8009-4c41-996b-fa5de29bb202 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:40.599392 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 727/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:42.708234 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23155d40-f5fd-49d1-9a7c-0ad8044396d9 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:42.708768 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23155d40-f5fd-49d1-9a7c-0ad8044396d9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:42.708768 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23155d40-f5fd-49d1-9a7c-0ad8044396d9 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:42.710142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 09:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:42.716397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92155811-b356-49b7-aef8-61c980d5682f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:42.716899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92155811-b356-49b7-aef8-61c980d5682f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:42.717451 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92155811-b356-49b7-aef8-61c980d5682f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:42.786003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-92155811-b356-49b7-aef8-61c980d5682f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:42.787222 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-92155811-b356-49b7-aef8-61c980d5682f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:42.807133 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:42.891141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 09:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:03:42.930633 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23155d40-f5fd-49d1-9a7c-0ad8044396d9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:42.932310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-23155d40-f5fd-49d1-9a7c-0ad8044396d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:43.034990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23155d40-f5fd-49d1-9a7c-0ad8044396d9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:43.106693 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:43.114108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a60f9b-a334-49bc-afd2-75e385bb2c83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:43.114763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a60f9b-a334-49bc-afd2-75e385bb2c83 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:43.115076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a60f9b-a334-49bc-afd2-75e385bb2c83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:43.115359 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a60f9b-a334-49bc-afd2-75e385bb2c83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:43.117040 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b3a60f9b-a334-49bc-afd2-75e385bb2c83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:43.230338 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a60f9b-a334-49bc-afd2-75e385bb2c83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:43.231702 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a60f9b-a334-49bc-afd2-75e385bb2c83 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:43.251866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 739/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:45.407506 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b8708-fb83-4968-84f6-88f544adb328 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:45.408157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b8708-fb83-4968-84f6-88f544adb328 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:45.408157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b8708-fb83-4968-84f6-88f544adb328 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:45.409484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 09:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:03:45.414961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b81544-c479-42d7-91dc-173ee9747e05 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:45.416143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b81544-c479-42d7-91dc-173ee9747e05 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:45.416143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b81544-c479-42d7-91dc-173ee9747e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:45.486015 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e3b81544-c479-42d7-91dc-173ee9747e05 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:03:45.487269 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e3b81544-c479-42d7-91dc-173ee9747e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:45.541880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:45.592001 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 09:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:03:45.630296 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b8708-fb83-4968-84f6-88f544adb328 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:03:45.631973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ff7b8708-fb83-4968-84f6-88f544adb328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:45.743140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b8708-fb83-4968-84f6-88f544adb328 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:03:45.791996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:45.822602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753b2c6-ec4a-4199-9158-bed89e0a2ff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:03:45.823260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753b2c6-ec4a-4199-9158-bed89e0a2ff2 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:03:45.823543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753b2c6-ec4a-4199-9158-bed89e0a2ff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:03:45.823821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753b2c6-ec4a-4199-9158-bed89e0a2ff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:03:45.825562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e753b2c6-ec4a-4199-9158-bed89e0a2ff2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:03:45.938670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753b2c6-ec4a-4199-9158-bed89e0a2ff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:03:45.940092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753b2c6-ec4a-4199-9158-bed89e0a2ff2 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:03:45.959022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 730/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:03:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:03:55.590077 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=22 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:03:55.956579 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=66 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:04:06.029929 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9cd96-ba04-4d13-a9e5-ea811c32168b None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:06.030475 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9cd96-ba04-4d13-a9e5-ea811c32168b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:06.030475 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9cd96-ba04-4d13-a9e5-ea811c32168b None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:06.031772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 741/1471] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:04:06.037607 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3abe5-87c0-40e8-ae6e-e205808383d3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:06.038121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3abe5-87c0-40e8-ae6e-e205808383d3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:06.038241 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3abe5-87c0-40e8-ae6e-e205808383d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:06.106712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-55d3abe5-87c0-40e8-ae6e-e205808383d3 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:04:06.107957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-55d3abe5-87c0-40e8-ae6e-e205808383d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:06.206124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 731/1472] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:04:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:04:06.296856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c904e6-bcdd-4b79-bcd1-b54d072bf937 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:06.297170 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c904e6-bcdd-4b79-bcd1-b54d072bf937 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:06.297450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c904e6-bcdd-4b79-bcd1-b54d072bf937 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:06.298797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 742/1473] 10.222.0.13 () {54 vars in 966 bytes} [Thu Jul 21 09:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:04:06.303998 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c3bb5-a52e-456e-ad87-e85a3fdd278b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:06.304298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c3bb5-a52e-456e-ad87-e85a3fdd278b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:06.304582 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c3bb5-a52e-456e-ad87-e85a3fdd278b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:06.382557 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-391c3bb5-a52e-456e-ad87-e85a3fdd278b None None] MFA Rules not processed for user `e053b6b25a4e4a20a05e6ffa6683222b`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:04:06.383808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-391c3bb5-a52e-456e-ad87-e85a3fdd278b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:06.405233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:04:06.464027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 732/1474] 10.222.0.13 () {58 vars in 1063 bytes} [Thu Jul 21 09:04:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:04:06.471980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c904e6-bcdd-4b79-bcd1-b54d072bf937 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:04:06.473518 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d8c904e6-bcdd-4b79-bcd1-b54d072bf937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:06.582127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c904e6-bcdd-4b79-bcd1-b54d072bf937 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:04:06.656886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d49cc-ce92-4655-bb7b-081e812e4171 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e053b6b25a4e4a20a05e6ffa6683222b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:04:06.657531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d49cc-ce92-4655-bb7b-081e812e4171 service glance] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:06.657828 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d49cc-ce92-4655-bb7b-081e812e4171 service glance] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:06.658085 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d49cc-ce92-4655-bb7b-081e812e4171 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:06.659726 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b63d49cc-ce92-4655-bb7b-081e812e4171 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:06.716888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:04:06.777881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d49cc-ce92-4655-bb7b-081e812e4171 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:04:06.779337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d49cc-ce92-4655-bb7b-081e812e4171 service glance] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:04:06.798281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 743/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:04:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:04:08.956662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91723b-dcfa-4a36-ad4a-1ea7ca639f2e None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:08.957249 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91723b-dcfa-4a36-ad4a-1ea7ca639f2e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:08.957249 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91723b-dcfa-4a36-ad4a-1ea7ca639f2e None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:08.958602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 733/1476] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:04:08.964614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea88636-115f-42f2-a4e9-6f10f08f0995 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:08.965055 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea88636-115f-42f2-a4e9-6f10f08f0995 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:08.965159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea88636-115f-42f2-a4e9-6f10f08f0995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:09.031252 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3ea88636-115f-42f2-a4e9-6f10f08f0995 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:04:09.032430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3ea88636-115f-42f2-a4e9-6f10f08f0995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:09.129866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 744/1477] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:04:09.153349 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91723b-dcfa-4a36-ad4a-1ea7ca639f2e None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:04:09.154941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3a91723b-dcfa-4a36-ad4a-1ea7ca639f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:09.286489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91723b-dcfa-4a36-ad4a-1ea7ca639f2e None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:04:09.331101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:04:09.377468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67907bd0-51df-44c0-9976-8ccfea13aaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:04:09.378135 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67907bd0-51df-44c0-9976-8ccfea13aaf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:09.378418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67907bd0-51df-44c0-9976-8ccfea13aaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:09.378692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67907bd0-51df-44c0-9976-8ccfea13aaf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:09.379422 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67907bd0-51df-44c0-9976-8ccfea13aaf2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:04:09.381298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-67907bd0-51df-44c0-9976-8ccfea13aaf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 09:04:09.381298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 21 09:04:09.381298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: Jul 21 09:04:09.382471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67907bd0-51df-44c0-9976-8ccfea13aaf2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:04:09.402751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 734/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jul 21 09:04:09 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:04:11.576795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921be249-078f-4a12-bde9-b502f5339719 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:11.577324 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921be249-078f-4a12-bde9-b502f5339719 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:11.577324 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921be249-078f-4a12-bde9-b502f5339719 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:11.578676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 745/1479] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:04:11.584309 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa8680-6cb9-4513-af9b-6a7f23cd28e7 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:11.584702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa8680-6cb9-4513-af9b-6a7f23cd28e7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:11.584867 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa8680-6cb9-4513-af9b-6a7f23cd28e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:11.652387 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-35aa8680-6cb9-4513-af9b-6a7f23cd28e7 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:04:11.653616 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-35aa8680-6cb9-4513-af9b-6a7f23cd28e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:11.756079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 735/1480] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:04:11.800057 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921be249-078f-4a12-bde9-b502f5339719 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:04:11.801698 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-921be249-078f-4a12-bde9-b502f5339719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:11.812705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:04:11.965626 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921be249-078f-4a12-bde9-b502f5339719 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:04:12.045247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:04:12.080942 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac2170-c39b-4915-b6fe-4ec1b6ddac10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:04:12.081594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac2170-c39b-4915-b6fe-4ec1b6ddac10 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:12.081874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac2170-c39b-4915-b6fe-4ec1b6ddac10 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:12.082147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac2170-c39b-4915-b6fe-4ec1b6ddac10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:12.083789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-73ac2170-c39b-4915-b6fe-4ec1b6ddac10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:12.174383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac2170-c39b-4915-b6fe-4ec1b6ddac10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:04:12.176147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac2170-c39b-4915-b6fe-4ec1b6ddac10 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:04:12.194773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 746/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:04:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:04:14.447097 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e119154-1c81-482d-b139-bbf1b96c9afa None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:14.447768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e119154-1c81-482d-b139-bbf1b96c9afa None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:14.447768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e119154-1c81-482d-b139-bbf1b96c9afa None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:14.449994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 736/1482] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:04:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:04:14.456134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199a504-a295-464d-b975-a1c55bee3aa0 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:14.456687 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199a504-a295-464d-b975-a1c55bee3aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:14.456687 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199a504-a295-464d-b975-a1c55bee3aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:14.523522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a199a504-a295-464d-b975-a1c55bee3aa0 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:04:14.524792 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a199a504-a295-464d-b975-a1c55bee3aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:14.574222 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:04:14.758538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 747/1483] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:04:14.795486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e119154-1c81-482d-b139-bbf1b96c9afa None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:04:14.797090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7e119154-1c81-482d-b139-bbf1b96c9afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:14.822766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:04:14.906550 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e119154-1c81-482d-b139-bbf1b96c9afa None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:04:14.984270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce475c31-7266-4ce9-9b4a-86f6910bff90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:04:14.984825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce475c31-7266-4ce9-9b4a-86f6910bff90 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:14.985108 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce475c31-7266-4ce9-9b4a-86f6910bff90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:14.985385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce475c31-7266-4ce9-9b4a-86f6910bff90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:14.987046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ce475c31-7266-4ce9-9b4a-86f6910bff90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:15.062994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:04:15.098877 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce475c31-7266-4ce9-9b4a-86f6910bff90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:04:15.100375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce475c31-7266-4ce9-9b4a-86f6910bff90 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:04:15.118242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 737/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:04:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:04:17.240375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed3c76-0df0-4739-b8df-26ee9ad9bb30 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:17.240951 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed3c76-0df0-4739-b8df-26ee9ad9bb30 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:17.240951 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed3c76-0df0-4739-b8df-26ee9ad9bb30 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:17.242899 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 748/1485] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:04:17.248725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7215eb-ddd3-4158-b051-763c9a4d1dc0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:17.249210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7215eb-ddd3-4158-b051-763c9a4d1dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:17.249311 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7215eb-ddd3-4158-b051-763c9a4d1dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:17.315366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-df7215eb-ddd3-4158-b051-763c9a4d1dc0 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:04:17.316590 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-df7215eb-ddd3-4158-b051-763c9a4d1dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:17.415182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 738/1486] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:04:17.454360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed3c76-0df0-4739-b8df-26ee9ad9bb30 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:04:17.455981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8aed3c76-0df0-4739-b8df-26ee9ad9bb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:17.563298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed3c76-0df0-4739-b8df-26ee9ad9bb30 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:04:17.639939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:04:17.642013 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c5264-6046-4006-b560-2da7eaabf0e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:04:17.642743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c5264-6046-4006-b560-2da7eaabf0e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:17.643221 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c5264-6046-4006-b560-2da7eaabf0e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:17.643708 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c5264-6046-4006-b560-2da7eaabf0e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:17.645611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-277c5264-6046-4006-b560-2da7eaabf0e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:17.766386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c5264-6046-4006-b560-2da7eaabf0e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:04:17.767828 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c5264-6046-4006-b560-2da7eaabf0e5 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:04:17.786236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 749/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:04:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:04:19.950349 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d3ad8-82f0-47e3-998f-f5677fc6a6f8 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:19.950914 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d3ad8-82f0-47e3-998f-f5677fc6a6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:19.950914 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d3ad8-82f0-47e3-998f-f5677fc6a6f8 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:19.952253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 739/1488] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:04:19.957703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cee6c-4db1-4586-8150-2ccea400dcfa None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:19.958177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cee6c-4db1-4586-8150-2ccea400dcfa None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:19.958277 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cee6c-4db1-4586-8150-2ccea400dcfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:20.021977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a95cee6c-4db1-4586-8150-2ccea400dcfa None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:04:20.023198 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a95cee6c-4db1-4586-8150-2ccea400dcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:20.077277 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:04:20.123443 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 750/1489] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:04:20.166315 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d3ad8-82f0-47e3-998f-f5677fc6a6f8 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:04:20.167844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9d1d3ad8-82f0-47e3-998f-f5677fc6a6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:20.205700 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:04:20.275216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d3ad8-82f0-47e3-998f-f5677fc6a6f8 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:04:20.349705 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac4626-8bd2-49db-ba8e-af45c4833671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:04:20.350358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac4626-8bd2-49db-ba8e-af45c4833671 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:20.350650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac4626-8bd2-49db-ba8e-af45c4833671 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:20.350924 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac4626-8bd2-49db-ba8e-af45c4833671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:20.353136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dfac4626-8bd2-49db-ba8e-af45c4833671 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:20.450454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:04:20.483711 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac4626-8bd2-49db-ba8e-af45c4833671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:04:20.485133 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac4626-8bd2-49db-ba8e-af45c4833671 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:04:20.504219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 740/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:04:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:04:22.770270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f3707-4764-41a0-a438-71ea79b51ae4 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:22.770892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f3707-4764-41a0-a438-71ea79b51ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:22.770892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f3707-4764-41a0-a438-71ea79b51ae4 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:22.772271 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 751/1491] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jul 21 09:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:04:22.778483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f67a21-0540-481d-bf87-22137e53d8d0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:22.778962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f67a21-0540-481d-bf87-22137e53d8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:22.779072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f67a21-0540-481d-bf87-22137e53d8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:22.848327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d9f67a21-0540-481d-bf87-22137e53d8d0 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:04:22.849557 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d9f67a21-0540-481d-bf87-22137e53d8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:22.951430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 741/1492] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jul 21 09:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:04:22.991790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f3707-4764-41a0-a438-71ea79b51ae4 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:04:22.993859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a67f3707-4764-41a0-a438-71ea79b51ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:23.105412 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f3707-4764-41a0-a438-71ea79b51ae4 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:04:23.187096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a11f18-99c5-486d-b308-34ffda437674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:04:23.187751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a11f18-99c5-486d-b308-34ffda437674 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:04:23.188069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a11f18-99c5-486d-b308-34ffda437674 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:04:23.188343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a11f18-99c5-486d-b308-34ffda437674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:04:23.190000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-56a11f18-99c5-486d-b308-34ffda437674 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:04:23.200430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:04:23.308417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a11f18-99c5-486d-b308-34ffda437674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:04:23.309781 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a11f18-99c5-486d-b308-34ffda437674 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:04:23.327945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 752/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:04:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:04:32.949617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=82 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:04:33.325688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=50 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:05:51.392624 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60675770-c2b6-471b-9b32-31b348467938 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:51.393307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60675770-c2b6-471b-9b32-31b348467938 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:51.393472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60675770-c2b6-471b-9b32-31b348467938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:51.494112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-60675770-c2b6-471b-9b32-31b348467938 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:05:51.495345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-60675770-c2b6-471b-9b32-31b348467938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:51.599247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 742/1494] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:05:51.607839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c9e75-09ae-4555-9748-306de8be84a8 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:51.608419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c9e75-09ae-4555-9748-306de8be84a8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:51.608419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c9e75-09ae-4555-9748-306de8be84a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:51.702340 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-030c9e75-09ae-4555-9748-306de8be84a8 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:05:51.703495 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-030c9e75-09ae-4555-9748-306de8be84a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:51.809331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 753/1495] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:05:51.814104 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60675770-c2b6-471b-9b32-31b348467938 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:05:51.815310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-60675770-c2b6-471b-9b32-31b348467938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:51.828320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60675770-c2b6-471b-9b32-31b348467938 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:05:51.891790 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:05:51.926428 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf1c93-ad9c-4043-9c14-964d68831714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:05:51.927137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf1c93-ad9c-4043-9c14-964d68831714 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:51.927674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf1c93-ad9c-4043-9c14-964d68831714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:51.927978 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf1c93-ad9c-4043-9c14-964d68831714 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:51.928704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf1c93-ad9c-4043-9c14-964d68831714 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:05:51.929019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf1c93-ad9c-4043-9c14-964d68831714 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:05:51.930081 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-3fbf1c93-ad9c-4043-9c14-964d68831714 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 09:05:51.930081 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 21 09:05:51.930081 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: Jul 21 09:05:51.931006 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf1c93-ad9c-4043-9c14-964d68831714 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:05:51.942906 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 743/1496] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:05:51.951721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c9e75-09ae-4555-9748-306de8be84a8 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:05:51.961972 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c9e75-09ae-4555-9748-306de8be84a8 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:05:52.024195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:05:52.057504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bbc3c-de50-44b1-b9d4-1afc862fabd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:05:52.058151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bbc3c-de50-44b1-b9d4-1afc862fabd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:52.058428 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bbc3c-de50-44b1-b9d4-1afc862fabd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:52.058699 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bbc3c-de50-44b1-b9d4-1afc862fabd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:52.059380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906bbc3c-de50-44b1-b9d4-1afc862fabd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:05:52.061136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: WARNING py.warnings [None req-906bbc3c-de50-44b1-b9d4-1afc862fabd7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 09:05:52.061136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: warnings.warn(msg) Jul 21 09:05:52.061136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: Jul 21 09:05:52.062349 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906bbc3c-de50-44b1-b9d4-1afc862fabd7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:05:52.119159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 754/1497] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:05:51 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:05:52.123573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf1c93-ad9c-4043-9c14-964d68831714 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:05:52.125129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3fbf1c93-ad9c-4043-9c14-964d68831714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:52.159104 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf1c93-ad9c-4043-9c14-964d68831714 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:05:52.254291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100465b-6d31-4ee1-b536-f0aadacaea55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:05:52.255456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100465b-6d31-4ee1-b536-f0aadacaea55 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:52.255738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100465b-6d31-4ee1-b536-f0aadacaea55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:52.256167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100465b-6d31-4ee1-b536-f0aadacaea55 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:52.256809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100465b-6d31-4ee1-b536-f0aadacaea55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:05:52.257835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-2100465b-6d31-4ee1-b536-f0aadacaea55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 09:05:52.257835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 21 09:05:52.257835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: Jul 21 09:05:52.259053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100465b-6d31-4ee1-b536-f0aadacaea55 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:05:52.299271 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 744/1498] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:05:52 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:05:52.304213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bbc3c-de50-44b1-b9d4-1afc862fabd7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:05:52.305746 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-906bbc3c-de50-44b1-b9d4-1afc862fabd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:52.368735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bbc3c-de50-44b1-b9d4-1afc862fabd7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:05:52.458063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:05:52.468871 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51829ef-524b-4140-92ef-a18302f06fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:05:52.469549 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51829ef-524b-4140-92ef-a18302f06fee admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:52.469834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51829ef-524b-4140-92ef-a18302f06fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:52.470109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51829ef-524b-4140-92ef-a18302f06fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:52.470831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51829ef-524b-4140-92ef-a18302f06fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:05:52.471854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: WARNING py.warnings [None req-b51829ef-524b-4140-92ef-a18302f06fee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 09:05:52.471854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: warnings.warn(msg) Jul 21 09:05:52.471854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: Jul 21 09:05:52.472776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51829ef-524b-4140-92ef-a18302f06fee admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:05:52.485275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 755/1499] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:05:52.490459 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100465b-6d31-4ee1-b536-f0aadacaea55 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:05:52.492033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2100465b-6d31-4ee1-b536-f0aadacaea55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:52.503594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100465b-6d31-4ee1-b536-f0aadacaea55 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:05:52.530071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:05:52.602575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685b9bd-4e54-49c1-bb2e-6c4446824e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:05:52.603242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685b9bd-4e54-49c1-bb2e-6c4446824e71 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:52.603526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685b9bd-4e54-49c1-bb2e-6c4446824e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:52.603804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685b9bd-4e54-49c1-bb2e-6c4446824e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:52.604571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6685b9bd-4e54-49c1-bb2e-6c4446824e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:05:52.605637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-6685b9bd-4e54-49c1-bb2e-6c4446824e71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 09:05:52.605637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 21 09:05:52.605637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: Jul 21 09:05:52.606532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6685b9bd-4e54-49c1-bb2e-6c4446824e71 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:05:52.619704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 745/1500] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:05:52.624093 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51829ef-524b-4140-92ef-a18302f06fee admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:05:52.625623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b51829ef-524b-4140-92ef-a18302f06fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:52.658505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51829ef-524b-4140-92ef-a18302f06fee admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:05:52.754430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3300f-fb45-4ad2-80d2-87c508bb556b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:05:52.755880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3300f-fb45-4ad2-80d2-87c508bb556b admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:52.756187 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3300f-fb45-4ad2-80d2-87c508bb556b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:52.756459 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3300f-fb45-4ad2-80d2-87c508bb556b admin admin] PATH_INFO: `/v3/projects/0fb35284a6a24868aed0e7bcfd7e8303/users/d155ca7c1e7b4f739688a19f344d47fc/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:52.778501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3300f-fb45-4ad2-80d2-87c508bb556b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb35284a6a24868aed0e7bcfd7e8303, user_id=d155ca7c1e7b4f739688a19f344d47fc, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:05:52.779529 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: WARNING py.warnings [None req-23f3300f-fb45-4ad2-80d2-87c508bb556b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 09:05:52.779529 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: warnings.warn(msg) Jul 21 09:05:52.779529 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: Jul 21 09:05:52.781565 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3300f-fb45-4ad2-80d2-87c508bb556b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:05:52.805829 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 756/1501] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:05:52 2022] PUT /identity/v3/projects/0fb35284a6a24868aed0e7bcfd7e8303/users/d155ca7c1e7b4f739688a19f344d47fc/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:05:52.819229 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13768b9-4cec-4d30-b0ef-204c621f63aa None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:52.819521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13768b9-4cec-4d30-b0ef-204c621f63aa None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:52.819801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13768b9-4cec-4d30-b0ef-204c621f63aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:52.874764 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b13768b9-4cec-4d30-b0ef-204c621f63aa None None] MFA Rules not processed for user `d155ca7c1e7b4f739688a19f344d47fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:05:52.876010 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b13768b9-4cec-4d30-b0ef-204c621f63aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:52.955221 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 746/1502] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:05:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:05:52.968661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8e639-4542-497c-9aae-3d158a877e7f None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:52.968959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8e639-4542-497c-9aae-3d158a877e7f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:52.969242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8e639-4542-497c-9aae-3d158a877e7f None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:52.970612 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 757/1503] 10.222.0.13 () {54 vars in 964 bytes} [Thu Jul 21 09:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:05:52.976032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4003030-8219-425f-b08f-5aa2cf1947b5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:52.976329 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4003030-8219-425f-b08f-5aa2cf1947b5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:52.976603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4003030-8219-425f-b08f-5aa2cf1947b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:53.008166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:05:53.052918 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f4003030-8219-425f-b08f-5aa2cf1947b5 None None] MFA Rules not processed for user `8a3b3421e9d8427388f4fd2be5a601cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:05:53.053683 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f4003030-8219-425f-b08f-5aa2cf1947b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:53.135799 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 747/1504] 10.222.0.13 () {58 vars in 1061 bytes} [Thu Jul 21 09:05:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:05:53.143709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8e639-4542-497c-9aae-3d158a877e7f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:05:53.145287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-87d8e639-4542-497c-9aae-3d158a877e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:53.195675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:05:53.252844 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8e639-4542-497c-9aae-3d158a877e7f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:05:53.327644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19e5ec-085f-43d8-9ab1-357dcd654436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:05:53.328236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19e5ec-085f-43d8-9ab1-357dcd654436 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:53.328526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19e5ec-085f-43d8-9ab1-357dcd654436 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:53.328798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19e5ec-085f-43d8-9ab1-357dcd654436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:53.330426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4d19e5ec-085f-43d8-9ab1-357dcd654436 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:53.430503 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19e5ec-085f-43d8-9ab1-357dcd654436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:05:53.431991 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19e5ec-085f-43d8-9ab1-357dcd654436 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:05:53.435869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:05:53.451455 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 758/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:05:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:05:53.495546 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2876b-60f8-4e1a-8c16-734d8d0e296b None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:53.496072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2876b-60f8-4e1a-8c16-734d8d0e296b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:53.496169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2876b-60f8-4e1a-8c16-734d8d0e296b None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:53.497415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 748/1506] 10.222.0.13 () {52 vars in 829 bytes} [Thu Jul 21 09:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:05:53.501297 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19e5ec-085f-43d8-9ab1-357dcd654436 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:05:53.502508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4d19e5ec-085f-43d8-9ab1-357dcd654436 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:53.611244 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19e5ec-085f-43d8-9ab1-357dcd654436 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:05:53.719676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f80536-86ba-41b3-8566-94ea093b58c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:05:53.720615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f80536-86ba-41b3-8566-94ea093b58c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:53.720899 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f80536-86ba-41b3-8566-94ea093b58c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:53.721173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f80536-86ba-41b3-8566-94ea093b58c7 admin admin] PATH_INFO: `/v3/users/d155ca7c1e7b4f739688a19f344d47fc` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:53.735087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f80536-86ba-41b3-8566-94ea093b58c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d155ca7c1e7b4f739688a19f344d47fc)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:05:53.736166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: WARNING py.warnings [None req-52f80536-86ba-41b3-8566-94ea093b58c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 09:05:53.736166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: warnings.warn(msg) Jul 21 09:05:53.736166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: Jul 21 09:05:53.737394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f80536-86ba-41b3-8566-94ea093b58c7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:05:53.747481 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:05:53.807151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-52f80536-86ba-41b3-8566-94ea093b58c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd155ca7c1e7b4f739688a19f344d47fc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:05:53.807699 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-52f80536-86ba-41b3-8566-94ea093b58c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd155ca7c1e7b4f739688a19f344d47fc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:05:53.817334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-52f80536-86ba-41b3-8566-94ea093b58c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd155ca7c1e7b4f739688a19f344d47fc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:05:53.838996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-52f80536-86ba-41b3-8566-94ea093b58c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd155ca7c1e7b4f739688a19f344d47fc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:05:53.851858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-52f80536-86ba-41b3-8566-94ea093b58c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd155ca7c1e7b4f739688a19f344d47fc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:05:53.857958 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 759/1507] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:05:53 2022] DELETE /identity/v3/users/d155ca7c1e7b4f739688a19f344d47fc => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:05:53.862459 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad118f-d8e9-4040-bb27-6f1867864738 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:53.862685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad118f-d8e9-4040-bb27-6f1867864738 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:53.862981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad118f-d8e9-4040-bb27-6f1867864738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:53.920300 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e2ad118f-d8e9-4040-bb27-6f1867864738 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:05:53.921521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e2ad118f-d8e9-4040-bb27-6f1867864738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:53.999755 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:05:54.012001 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 749/1508] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:05:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:05:54.022215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f80536-86ba-41b3-8566-94ea093b58c7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:05:54.023461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-52f80536-86ba-41b3-8566-94ea093b58c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:54.126041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f80536-86ba-41b3-8566-94ea093b58c7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:05:54.183012 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:05:54.204742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7993cf-f9ac-4ba9-8d93-96d42ae16f6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:05:54.205397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7993cf-f9ac-4ba9-8d93-96d42ae16f6a service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:54.205680 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7993cf-f9ac-4ba9-8d93-96d42ae16f6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:54.205954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7993cf-f9ac-4ba9-8d93-96d42ae16f6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:54.207629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fe7993cf-f9ac-4ba9-8d93-96d42ae16f6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:54.330437 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7993cf-f9ac-4ba9-8d93-96d42ae16f6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:05:54.331896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7993cf-f9ac-4ba9-8d93-96d42ae16f6a service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:05:54.350932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 760/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:05:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:05:54.806688 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad118f-d8e9-4040-bb27-6f1867864738 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:05:54.808011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e2ad118f-d8e9-4040-bb27-6f1867864738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:05:54.820402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad118f-d8e9-4040-bb27-6f1867864738 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:05:54.933032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b584f58-3efb-4089-839d-6091f1ecbd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:05:54.933892 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b584f58-3efb-4089-839d-6091f1ecbd03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:05:54.934170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b584f58-3efb-4089-839d-6091f1ecbd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:05:54.934438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b584f58-3efb-4089-839d-6091f1ecbd03 admin admin] PATH_INFO: `/v3/projects/0fb35284a6a24868aed0e7bcfd7e8303` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:05:54.945299 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b584f58-3efb-4089-839d-6091f1ecbd03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fb35284a6a24868aed0e7bcfd7e8303)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:05:54.946351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-2b584f58-3efb-4089-839d-6091f1ecbd03 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 09:05:54.946351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 21 09:05:54.946351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: Jul 21 09:05:54.947669 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b584f58-3efb-4089-839d-6091f1ecbd03 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:05:55.007564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2b584f58-3efb-4089-839d-6091f1ecbd03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fb35284a6a24868aed0e7bcfd7e8303', 'request_id': 'req-2b584f58-3efb-4089-839d-6091f1ecbd03'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:05:55.023459 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2b584f58-3efb-4089-839d-6091f1ecbd03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fb35284a6a24868aed0e7bcfd7e8303', 'request_id': 'req-2b584f58-3efb-4089-839d-6091f1ecbd03'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:05:55.032861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2b584f58-3efb-4089-839d-6091f1ecbd03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fb35284a6a24868aed0e7bcfd7e8303', 'request_id': 'req-2b584f58-3efb-4089-839d-6091f1ecbd03'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:05:55.176064 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2b584f58-3efb-4089-839d-6091f1ecbd03 admin admin] The token cache is being invalidate because project 0fb35284a6a24868aed0e7bcfd7e8303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:05:55.176432 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2b584f58-3efb-4089-839d-6091f1ecbd03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:05:55.177986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2b584f58-3efb-4089-839d-6091f1ecbd03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:05:55.179380 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 750/1510] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:05:54 2022] DELETE /identity/v3/projects/0fb35284a6a24868aed0e7bcfd7e8303 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:06:04.348615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=62 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:06:05.173857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:07:35.273952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab68ea1-5402-45c6-8015-22cf50c59519 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:35.274623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab68ea1-5402-45c6-8015-22cf50c59519 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:35.274623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab68ea1-5402-45c6-8015-22cf50c59519 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:35.276004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 09:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 09:07:35.283238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6c74d-f51e-44e6-b609-9482581398c3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:35.283812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6c74d-f51e-44e6-b609-9482581398c3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:35.284193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6c74d-f51e-44e6-b609-9482581398c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:35.424022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5da6c74d-f51e-44e6-b609-9482581398c3 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:07:35.425381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5da6c74d-f51e-44e6-b609-9482581398c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:07:35.582431 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 09:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:07:35.755848 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab68ea1-5402-45c6-8015-22cf50c59519 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:07:35.758361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0ab68ea1-5402-45c6-8015-22cf50c59519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:07:35.925644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab68ea1-5402-45c6-8015-22cf50c59519 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:07:35.934800 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a6fbb-3601-42b8-afdc-6be795b28fb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:07:35.935939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a6fbb-3601-42b8-afdc-6be795b28fb5 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:35.936433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a6fbb-3601-42b8-afdc-6be795b28fb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:35.936932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a6fbb-3601-42b8-afdc-6be795b28fb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:35.952876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a6fbb-3601-42b8-afdc-6be795b28fb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:07:35.956105 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a6fbb-3601-42b8-afdc-6be795b28fb5 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:07:35.975502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 762/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:07:36.858324 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a4c09-6943-42b6-83fc-0c9948e62829 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:36.858905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a4c09-6943-42b6-83fc-0c9948e62829 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:36.858905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a4c09-6943-42b6-83fc-0c9948e62829 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:36.860366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 09:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 09:07:36.865874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34325ff-522c-4b14-89d2-af0b8eaf7a9a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:36.866423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34325ff-522c-4b14-89d2-af0b8eaf7a9a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:36.866566 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34325ff-522c-4b14-89d2-af0b8eaf7a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:36.889735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a34325ff-522c-4b14-89d2-af0b8eaf7a9a None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:07:36.890991 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a34325ff-522c-4b14-89d2-af0b8eaf7a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:07:36.900092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 09:07:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:07:37.062497 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a4c09-6943-42b6-83fc-0c9948e62829 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:07:37.078241 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a4c09-6943-42b6-83fc-0c9948e62829 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:07:37.084404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a3f2f-59f9-47b7-be0c-bd541796eaed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:07:37.085076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a3f2f-59f9-47b7-be0c-bd541796eaed service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:37.085361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a3f2f-59f9-47b7-be0c-bd541796eaed service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:37.085638 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a3f2f-59f9-47b7-be0c-bd541796eaed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:37.097011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a3f2f-59f9-47b7-be0c-bd541796eaed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:07:37.098818 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a3f2f-59f9-47b7-be0c-bd541796eaed service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:07:37.111076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 753/1516] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:07:37.167547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d6603-8d1c-4066-a5a8-9de8d566a78e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:37.168115 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d6603-8d1c-4066-a5a8-9de8d566a78e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:37.168615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d6603-8d1c-4066-a5a8-9de8d566a78e None None] PATH_INFO: `/v3` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:37.171283 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 764/1517] 10.222.0.36 () {54 vars in 925 bytes} [Thu Jul 21 09:07:37 2022] GET /identity/v3 => generated 255 bytes in 6 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:07:37.179391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e11fb-8da3-45ea-b195-c70bac79b64b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:37.180076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e11fb-8da3-45ea-b195-c70bac79b64b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:37.180667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e11fb-8da3-45ea-b195-c70bac79b64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:37.271029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-cd5e11fb-8da3-45ea-b195-c70bac79b64b None None] MFA Rules not processed for user `72f9ef6de66e456e800237e9f954e98d`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:07:37.272564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-cd5e11fb-8da3-45ea-b195-c70bac79b64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:07:37.299712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:07:37.370275 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 754/1518] 10.222.0.36 () {58 vars in 1015 bytes} [Thu Jul 21 09:07:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:07:46.489052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ad6cb-d9b4-4e9d-8e1a-e0994a06033c None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:46.489662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ad6cb-d9b4-4e9d-8e1a-e0994a06033c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:46.489662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ad6cb-d9b4-4e9d-8e1a-e0994a06033c None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:46.491255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 765/1519] 10.222.0.13 () {54 vars in 961 bytes} [Thu Jul 21 09:07:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:07:46.497361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc708d-3ca1-4f70-9e10-3da6a9253fcf None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:46.498061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc708d-3ca1-4f70-9e10-3da6a9253fcf None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:46.498061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc708d-3ca1-4f70-9e10-3da6a9253fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:46.519219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-09bc708d-3ca1-4f70-9e10-3da6a9253fcf None None] MFA Rules not processed for user `72f9ef6de66e456e800237e9f954e98d`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:07:46.520567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-09bc708d-3ca1-4f70-9e10-3da6a9253fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:07:46.529652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 755/1520] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Jul 21 09:07:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:07:46.542864 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ad6cb-d9b4-4e9d-8e1a-e0994a06033c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:07:46.559128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ad6cb-d9b4-4e9d-8e1a-e0994a06033c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:07:46.565338 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313e573-a89d-4d4e-9a10-1b1abd38afdb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:07:46.566005 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313e573-a89d-4d4e-9a10-1b1abd38afdb service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:46.566288 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313e573-a89d-4d4e-9a10-1b1abd38afdb service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:46.566565 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313e573-a89d-4d4e-9a10-1b1abd38afdb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:46.576096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313e573-a89d-4d4e-9a10-1b1abd38afdb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:07:46.577735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313e573-a89d-4d4e-9a10-1b1abd38afdb service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:07:46.590127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 766/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:07:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:07:48.429141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d91ad3-1330-474f-bfbd-964427b3ed1f None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:48.429769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d91ad3-1330-474f-bfbd-964427b3ed1f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:48.429769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d91ad3-1330-474f-bfbd-964427b3ed1f None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:48.431385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 09:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 09:07:48.437496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cfe87-d163-4607-ba9f-86d3dc65ea3d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:48.438634 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cfe87-d163-4607-ba9f-86d3dc65ea3d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:48.438634 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3cfe87-d163-4607-ba9f-86d3dc65ea3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:48.458854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-5f3cfe87-d163-4607-ba9f-86d3dc65ea3d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:07:48.460300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5f3cfe87-d163-4607-ba9f-86d3dc65ea3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:07:48.468934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 09:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:07:48.651445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d91ad3-1330-474f-bfbd-964427b3ed1f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:07:48.668346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d91ad3-1330-474f-bfbd-964427b3ed1f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:07:48.675289 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068d67a-9b6e-4add-a813-634250aafe18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:07:48.676050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068d67a-9b6e-4add-a813-634250aafe18 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:48.676407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068d67a-9b6e-4add-a813-634250aafe18 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:48.677067 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068d67a-9b6e-4add-a813-634250aafe18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:48.688587 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068d67a-9b6e-4add-a813-634250aafe18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:07:48.690564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068d67a-9b6e-4add-a813-634250aafe18 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:07:48.701505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 757/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:07:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:07:49.566900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cada85-3a24-4d47-85d5-dc17f8f20aef None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:49.567526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cada85-3a24-4d47-85d5-dc17f8f20aef None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:49.567526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cada85-3a24-4d47-85d5-dc17f8f20aef None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:49.568907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 09:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:07:49.574484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08890b52-49fd-4377-9fc0-2c1fa24b441f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:49.575286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08890b52-49fd-4377-9fc0-2c1fa24b441f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:49.575286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08890b52-49fd-4377-9fc0-2c1fa24b441f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:49.595433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-08890b52-49fd-4377-9fc0-2c1fa24b441f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:07:49.596720 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-08890b52-49fd-4377-9fc0-2c1fa24b441f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:07:49.605209 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 09:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:07:49.763555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cada85-3a24-4d47-85d5-dc17f8f20aef None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:07:49.780672 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cada85-3a24-4d47-85d5-dc17f8f20aef None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:07:49.787899 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0ab96-b67d-4921-a341-5d4360631c59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:07:49.788718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0ab96-b67d-4921-a341-5d4360631c59 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:49.789007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0ab96-b67d-4921-a341-5d4360631c59 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:49.789286 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0ab96-b67d-4921-a341-5d4360631c59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:49.800730 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0ab96-b67d-4921-a341-5d4360631c59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:07:49.802993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0ab96-b67d-4921-a341-5d4360631c59 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:07:49.814973 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 769/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:07:54.167701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56eaf9-4922-430a-8b2c-d28795ab521e None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:54.168376 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56eaf9-4922-430a-8b2c-d28795ab521e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:54.168664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56eaf9-4922-430a-8b2c-d28795ab521e None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:54.170343 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 09:07:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:07:54.177098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1ebea-4f6e-4458-a1a1-0881868e9023 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:54.177844 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1ebea-4f6e-4458-a1a1-0881868e9023 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:54.177844 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1ebea-4f6e-4458-a1a1-0881868e9023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:54.206594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-dbc1ebea-4f6e-4458-a1a1-0881868e9023 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:07:54.207909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-dbc1ebea-4f6e-4458-a1a1-0881868e9023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:07:54.216756 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 09:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:07:54.307863 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56eaf9-4922-430a-8b2c-d28795ab521e None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:07:54.309782 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4a56eaf9-4922-430a-8b2c-d28795ab521e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:07:54.410037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56eaf9-4922-430a-8b2c-d28795ab521e None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:07:54.414023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d1103-07bc-471d-a75e-75b3129ded31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:07:54.414697 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d1103-07bc-471d-a75e-75b3129ded31 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:54.414989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d1103-07bc-471d-a75e-75b3129ded31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:54.415269 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d1103-07bc-471d-a75e-75b3129ded31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:54.427549 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d1103-07bc-471d-a75e-75b3129ded31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:07:54.428988 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d1103-07bc-471d-a75e-75b3129ded31 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:07:54.441797 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 760/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:07:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:07:56.772845 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cf742-8e5a-45de-8ab3-79fdaf6cba67 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:56.773431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cf742-8e5a-45de-8ab3-79fdaf6cba67 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:56.773581 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1cf742-8e5a-45de-8ab3-79fdaf6cba67 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:56.775119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 09:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 09:07:56.781508 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbac9e6-008b-4d1b-a8bc-0abff77f9317 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:56.782385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbac9e6-008b-4d1b-a8bc-0abff77f9317 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:56.782546 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbac9e6-008b-4d1b-a8bc-0abff77f9317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:56.803962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5dbac9e6-008b-4d1b-a8bc-0abff77f9317 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:07:56.805483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5dbac9e6-008b-4d1b-a8bc-0abff77f9317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:07:56.815702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 09:07:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:07:56.856117 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1cf742-8e5a-45de-8ab3-79fdaf6cba67 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:07:56.858051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ad1cf742-8e5a-45de-8ab3-79fdaf6cba67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:07:56.872715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1cf742-8e5a-45de-8ab3-79fdaf6cba67 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:07:56.877226 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ecb3b-9cff-47d1-b050-d453022fa565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:07:56.877989 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ecb3b-9cff-47d1-b050-d453022fa565 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:07:56.878396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ecb3b-9cff-47d1-b050-d453022fa565 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:07:56.878736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ecb3b-9cff-47d1-b050-d453022fa565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:07:56.890206 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ecb3b-9cff-47d1-b050-d453022fa565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:07:56.891842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ecb3b-9cff-47d1-b050-d453022fa565 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:07:56.903066 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 772/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:07:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:08:01.010045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35b650-4e00-4c7c-87fd-60aa17f294cc None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:08:01.010860 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35b650-4e00-4c7c-87fd-60aa17f294cc None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:08:01.010860 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35b650-4e00-4c7c-87fd-60aa17f294cc None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:08:01.012684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 09:08:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:08:01.021128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9913db-93b3-4e49-8c1b-5b752293a643 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:08:01.021728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9913db-93b3-4e49-8c1b-5b752293a643 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:08:01.021728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9913db-93b3-4e49-8c1b-5b752293a643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:08:01.043000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1a9913db-93b3-4e49-8c1b-5b752293a643 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:08:01.044439 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1a9913db-93b3-4e49-8c1b-5b752293a643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:08:01.054988 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 09:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:08:01.404830 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35b650-4e00-4c7c-87fd-60aa17f294cc None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:08:01.407424 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6b35b650-4e00-4c7c-87fd-60aa17f294cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:08:01.505102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:08:01.533834 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35b650-4e00-4c7c-87fd-60aa17f294cc None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:08:01.540192 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6de1e-826f-415e-a79d-2a97d7560a7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:08:01.541632 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6de1e-826f-415e-a79d-2a97d7560a7a service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:08:01.542175 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6de1e-826f-415e-a79d-2a97d7560a7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:08:01.542750 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6de1e-826f-415e-a79d-2a97d7560a7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:08:01.560760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6de1e-826f-415e-a79d-2a97d7560a7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:08:01.564177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6de1e-826f-415e-a79d-2a97d7560a7a service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:08:01.581450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 763/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:08:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:08:03.877677 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa9b52-f85c-476a-ad13-24b7b5a72969 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:08:03.878259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa9b52-f85c-476a-ad13-24b7b5a72969 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:08:03.878259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa9b52-f85c-476a-ad13-24b7b5a72969 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:08:03.879627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 09:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:08:03.885602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf070b74-6a54-43dd-bd09-183ac80c6993 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:08:03.886189 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf070b74-6a54-43dd-bd09-183ac80c6993 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:08:03.886189 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf070b74-6a54-43dd-bd09-183ac80c6993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:08:03.905406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-bf070b74-6a54-43dd-bd09-183ac80c6993 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:08:03.907387 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bf070b74-6a54-43dd-bd09-183ac80c6993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:08:03.916889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 09:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:08:04.244335 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa9b52-f85c-476a-ad13-24b7b5a72969 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:08:04.246116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ddfa9b52-f85c-476a-ad13-24b7b5a72969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:08:04.261539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa9b52-f85c-476a-ad13-24b7b5a72969 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:08:04.265436 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc46475-c1e3-4b38-a8ad-db2b498a5a58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:08:04.266142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc46475-c1e3-4b38-a8ad-db2b498a5a58 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:08:04.266429 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc46475-c1e3-4b38-a8ad-db2b498a5a58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:08:04.266711 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc46475-c1e3-4b38-a8ad-db2b498a5a58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:08:04.277179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc46475-c1e3-4b38-a8ad-db2b498a5a58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:08:04.278679 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc46475-c1e3-4b38-a8ad-db2b498a5a58 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:08:04.290061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 775/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:08:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:08:13.900025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=11 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:08:14.284912 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:41.098923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b55d77-1e9e-4778-ac2e-4bdd4b9861e8 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.099854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b55d77-1e9e-4778-ac2e-4bdd4b9861e8 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.099854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b55d77-1e9e-4778-ac2e-4bdd4b9861e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.143562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-79b55d77-1e9e-4778-ac2e-4bdd4b9861e8 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:41.146016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-79b55d77-1e9e-4778-ac2e-4bdd4b9861e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:41.166663 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 765/1540] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:41.177800 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dee3f-bf51-4395-8230-9f49275d499f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.178725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dee3f-bf51-4395-8230-9f49275d499f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.179493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dee3f-bf51-4395-8230-9f49275d499f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.226931 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-954dee3f-bf51-4395-8230-9f49275d499f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:41.229152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-954dee3f-bf51-4395-8230-9f49275d499f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:41.250961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 776/1541] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:41.256836 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b55d77-1e9e-4778-ac2e-4bdd4b9861e8 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:41.282870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b55d77-1e9e-4778-ac2e-4bdd4b9861e8 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:41.297037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1915e7-9097-4c01-be7a-d0899708473c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:41.297037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1915e7-9097-4c01-be7a-d0899708473c admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.297037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1915e7-9097-4c01-be7a-d0899708473c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.297037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1915e7-9097-4c01-be7a-d0899708473c admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.301989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1915e7-9097-4c01-be7a-d0899708473c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:09:41.302531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1915e7-9097-4c01-be7a-d0899708473c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:41.312199 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1915e7-9097-4c01-be7a-d0899708473c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:41.344200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 766/1542] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 09:09:41.353596 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dee3f-bf51-4395-8230-9f49275d499f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:41.366125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3faa93-9ce7-4723-ae76-2057ef8a0b54 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.366628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3faa93-9ce7-4723-ae76-2057ef8a0b54 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.367086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3faa93-9ce7-4723-ae76-2057ef8a0b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.424540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dee3f-bf51-4395-8230-9f49275d499f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:41.445029 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb21dbb-b3d4-48d0-aec8-d38c0bfab31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:41.446168 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb21dbb-b3d4-48d0-aec8-d38c0bfab31f admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.446832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb21dbb-b3d4-48d0-aec8-d38c0bfab31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.447973 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb21dbb-b3d4-48d0-aec8-d38c0bfab31f admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.448517 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb21dbb-b3d4-48d0-aec8-d38c0bfab31f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:41.459538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-be3faa93-9ce7-4723-ae76-2057ef8a0b54 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:41.461706 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-be3faa93-9ce7-4723-ae76-2057ef8a0b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:41.464116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb21dbb-b3d4-48d0-aec8-d38c0bfab31f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:41.486603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 767/1543] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:41.489038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e67364-f236-41fc-b47f-01536fdfee37 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.489701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e67364-f236-41fc-b47f-01536fdfee37 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.490193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e67364-f236-41fc-b47f-01536fdfee37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.544485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-42e67364-f236-41fc-b47f-01536fdfee37 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:41.549427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-42e67364-f236-41fc-b47f-01536fdfee37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:41.577586 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 777/1545] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:09:41 2022] POST /identity/v3/projects => generated 405 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:41.578019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 768/1545] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:41.579835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7ca9b-92ec-42ae-a235-e48dfe12a3c5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.581725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadef1e1-6d30-4779-a7e4-b1ef8142b36f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.582844 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadef1e1-6d30-4779-a7e4-b1ef8142b36f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.583067 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7ca9b-92ec-42ae-a235-e48dfe12a3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.583565 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7ca9b-92ec-42ae-a235-e48dfe12a3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.585685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadef1e1-6d30-4779-a7e4-b1ef8142b36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.623241 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-bea7ca9b-92ec-42ae-a235-e48dfe12a3c5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:41.627610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bea7ca9b-92ec-42ae-a235-e48dfe12a3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:41.629422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-cadef1e1-6d30-4779-a7e4-b1ef8142b36f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:41.630665 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cadef1e1-6d30-4779-a7e4-b1ef8142b36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:41.778646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 769/1546] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:41.782926 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a7b11-4afe-48e9-91ca-da9890221190 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.788249 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a7b11-4afe-48e9-91ca-da9890221190 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.789094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a7b11-4afe-48e9-91ca-da9890221190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.798372 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 778/1547] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:41.800560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadef1e1-6d30-4779-a7e4-b1ef8142b36f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:41.823637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b36a7b11-4afe-48e9-91ca-da9890221190 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:41.825800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b36a7b11-4afe-48e9-91ca-da9890221190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:41.828011 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadef1e1-6d30-4779-a7e4-b1ef8142b36f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:41.836327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 770/1548] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:41.837617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de43328-b0ee-4e37-b9cf-97ac1950235f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:41.838666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de43328-b0ee-4e37-b9cf-97ac1950235f admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.839192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de43328-b0ee-4e37-b9cf-97ac1950235f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.839853 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de43328-b0ee-4e37-b9cf-97ac1950235f admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.841681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de43328-b0ee-4e37-b9cf-97ac1950235f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:41.842674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe899554-8391-4f41-8473-67f396085204 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.842969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe899554-8391-4f41-8473-67f396085204 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.843247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe899554-8391-4f41-8473-67f396085204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.843703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: WARNING py.warnings [None req-4de43328-b0ee-4e37-b9cf-97ac1950235f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 09:09:41.843703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: warnings.warn(msg) Jul 21 09:09:41.843703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: Jul 21 09:09:41.845826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de43328-b0ee-4e37-b9cf-97ac1950235f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:41.873789 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-fe899554-8391-4f41-8473-67f396085204 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:41.875672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fe899554-8391-4f41-8473-67f396085204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:41.890812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 771/1549] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:41.890812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f12a4c-b79f-4af4-9cd4-7511243fa345 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.890812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f12a4c-b79f-4af4-9cd4-7511243fa345 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.890812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f12a4c-b79f-4af4-9cd4-7511243fa345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.898050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 779/1550] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:09:41 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:41.900625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8047499-0150-4543-b5c9-8ef36c6de959 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.901400 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8047499-0150-4543-b5c9-8ef36c6de959 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.902009 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8047499-0150-4543-b5c9-8ef36c6de959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.913624 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-56f12a4c-b79f-4af4-9cd4-7511243fa345 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:41.914934 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-56f12a4c-b79f-4af4-9cd4-7511243fa345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:41.925121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 772/1551] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:41.929946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0018837-fe55-4d03-a093-6fe1f968c156 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.930568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0018837-fe55-4d03-a093-6fe1f968c156 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.930938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0018837-fe55-4d03-a093-6fe1f968c156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.939487 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d8047499-0150-4543-b5c9-8ef36c6de959 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:41.941658 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d8047499-0150-4543-b5c9-8ef36c6de959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:41.953198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e0018837-fe55-4d03-a093-6fe1f968c156 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:41.955082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e0018837-fe55-4d03-a093-6fe1f968c156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:41.959539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 780/1552] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:41.960928 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8047499-0150-4543-b5c9-8ef36c6de959 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:41.963946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 773/1553] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:41.966010 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31066b43-2018-4759-a851-939e10b233ca None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.966362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31066b43-2018-4759-a851-939e10b233ca None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.966852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31066b43-2018-4759-a851-939e10b233ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.978381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8047499-0150-4543-b5c9-8ef36c6de959 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:41.984473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4f614-3806-44b6-8006-2ea250ccce9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:41.985172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4f614-3806-44b6-8006-2ea250ccce9b admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:41.985458 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4f614-3806-44b6-8006-2ea250ccce9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:41.985731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4f614-3806-44b6-8006-2ea250ccce9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:41.986516 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4f614-3806-44b6-8006-2ea250ccce9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:09:41.986819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4f614-3806-44b6-8006-2ea250ccce9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:41.987870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: WARNING py.warnings [None req-aad4f614-3806-44b6-8006-2ea250ccce9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 09:09:41.987870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: warnings.warn(msg) Jul 21 09:09:41.987870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: Jul 21 09:09:41.988649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-31066b43-2018-4759-a851-939e10b233ca None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:41.988816 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4f614-3806-44b6-8006-2ea250ccce9b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:41.990038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-31066b43-2018-4759-a851-939e10b233ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:42.002388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 774/1554] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:42.003808 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 781/1555] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:09:42.004327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31066b43-2018-4759-a851-939e10b233ca None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.004805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4f614-3806-44b6-8006-2ea250ccce9b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.019050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31066b43-2018-4759-a851-939e10b233ca None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.019356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4f614-3806-44b6-8006-2ea250ccce9b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.024602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e377e-ec13-4f3f-9732-65f05c396683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.024929 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d805a55-4ecb-4b74-9161-6fd282cf46c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.025285 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e377e-ec13-4f3f-9732-65f05c396683 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.025568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e377e-ec13-4f3f-9732-65f05c396683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.025838 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d805a55-4ecb-4b74-9161-6fd282cf46c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.026018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e377e-ec13-4f3f-9732-65f05c396683 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.026295 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d805a55-4ecb-4b74-9161-6fd282cf46c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.026587 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e377e-ec13-4f3f-9732-65f05c396683 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:09:42.026760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d805a55-4ecb-4b74-9161-6fd282cf46c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.026912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e377e-ec13-4f3f-9732-65f05c396683 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.028522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d805a55-4ecb-4b74-9161-6fd282cf46c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.029230 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e377e-ec13-4f3f-9732-65f05c396683 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.030767 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d805a55-4ecb-4b74-9161-6fd282cf46c3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.044288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 775/1556] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:09:42.049296 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e377e-ec13-4f3f-9732-65f05c396683 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.049522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 782/1557] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:42.049522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facab1c9-6f55-4450-bbba-cc5f0051596f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.049522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facab1c9-6f55-4450-bbba-cc5f0051596f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.049522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facab1c9-6f55-4450-bbba-cc5f0051596f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.066126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e377e-ec13-4f3f-9732-65f05c396683 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.071622 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a14e0-b0d4-491f-ab25-9aafb95d625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.072459 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a14e0-b0d4-491f-ab25-9aafb95d625e admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.072750 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a14e0-b0d4-491f-ab25-9aafb95d625e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.073031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a14e0-b0d4-491f-ab25-9aafb95d625e admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.073794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a14e0-b0d4-491f-ab25-9aafb95d625e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:09:42.074107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a14e0-b0d4-491f-ab25-9aafb95d625e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.075675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-facab1c9-6f55-4450-bbba-cc5f0051596f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:42.076486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a14e0-b0d4-491f-ab25-9aafb95d625e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.077184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-facab1c9-6f55-4450-bbba-cc5f0051596f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:42.086354 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 783/1558] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:42.087775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377edb5-8f38-43df-a135-b963b3a0f538 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.088109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377edb5-8f38-43df-a135-b963b3a0f538 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.088386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377edb5-8f38-43df-a135-b963b3a0f538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.093070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 776/1559] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:09:42.094508 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a14e0-b0d4-491f-ab25-9aafb95d625e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.117476 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0377edb5-8f38-43df-a135-b963b3a0f538 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:42.118988 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0377edb5-8f38-43df-a135-b963b3a0f538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:42.119662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a14e0-b0d4-491f-ab25-9aafb95d625e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.128512 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cb72f-0e24-4540-a2af-8dba8078381f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.129566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cb72f-0e24-4540-a2af-8dba8078381f admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.129771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 784/1560] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:42.130076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cb72f-0e24-4540-a2af-8dba8078381f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.130566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cb72f-0e24-4540-a2af-8dba8078381f admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.131479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678b0f1-6f7b-4ab1-a883-d463da6adb56 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.131950 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678b0f1-6f7b-4ab1-a883-d463da6adb56 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.132164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3cb72f-0e24-4540-a2af-8dba8078381f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:09:42.132473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678b0f1-6f7b-4ab1-a883-d463da6adb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.132661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3cb72f-0e24-4540-a2af-8dba8078381f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.135769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3cb72f-0e24-4540-a2af-8dba8078381f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.155524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1678b0f1-6f7b-4ab1-a883-d463da6adb56 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:42.156832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1678b0f1-6f7b-4ab1-a883-d463da6adb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:42.165021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 777/1561] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:09:42.170201 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 785/1562] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:42.170462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cb72f-0e24-4540-a2af-8dba8078381f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.170556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678b0f1-6f7b-4ab1-a883-d463da6adb56 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.183109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678b0f1-6f7b-4ab1-a883-d463da6adb56 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.187811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cb72f-0e24-4540-a2af-8dba8078381f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.188996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2be005-dc9a-4da4-aefc-7244fa34e827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.189707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2be005-dc9a-4da4-aefc-7244fa34e827 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.189989 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2be005-dc9a-4da4-aefc-7244fa34e827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.190272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2be005-dc9a-4da4-aefc-7244fa34e827 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.191148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2be005-dc9a-4da4-aefc-7244fa34e827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.192870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2be005-dc9a-4da4-aefc-7244fa34e827 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.196070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c1363-753f-4509-b6d5-c4ba6ca91c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.197079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c1363-753f-4509-b6d5-c4ba6ca91c04 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.197535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c1363-753f-4509-b6d5-c4ba6ca91c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.197989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c1363-753f-4509-b6d5-c4ba6ca91c04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.199193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c1363-753f-4509-b6d5-c4ba6ca91c04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.201656 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-038c1363-753f-4509-b6d5-c4ba6ca91c04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 09:09:42.201656 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 21 09:09:42.201656 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: Jul 21 09:09:42.203858 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c1363-753f-4509-b6d5-c4ba6ca91c04 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.212797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 786/1563] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:42.212797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2be005-dc9a-4da4-aefc-7244fa34e827 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.223999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2be005-dc9a-4da4-aefc-7244fa34e827 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.231104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a9f61-5465-4247-9eec-800dc6228ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.232854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a9f61-5465-4247-9eec-800dc6228ced admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.233207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a9f61-5465-4247-9eec-800dc6228ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.233540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a9f61-5465-4247-9eec-800dc6228ced admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.234345 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a9f61-5465-4247-9eec-800dc6228ced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.237380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a9f61-5465-4247-9eec-800dc6228ced admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.253495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 778/1564] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:09:42 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:42.254993 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983551d8-25a9-4349-a49e-4832ec248c39 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.255285 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983551d8-25a9-4349-a49e-4832ec248c39 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.255564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983551d8-25a9-4349-a49e-4832ec248c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.280131 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 787/1565] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:09:42 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:42.281021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a9f61-5465-4247-9eec-800dc6228ced admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.282153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-983551d8-25a9-4349-a49e-4832ec248c39 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:42.284105 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-983551d8-25a9-4349-a49e-4832ec248c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:42.318778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:42.336972 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:42.366387 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a9f61-5465-4247-9eec-800dc6228ced admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.374040 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704b28d-106b-477b-aaa5-04d8f16f25a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.374821 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704b28d-106b-477b-aaa5-04d8f16f25a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.375198 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704b28d-106b-477b-aaa5-04d8f16f25a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.375536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704b28d-106b-477b-aaa5-04d8f16f25a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.376666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704b28d-106b-477b-aaa5-04d8f16f25a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.379141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704b28d-106b-477b-aaa5-04d8f16f25a4 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.386390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 779/1566] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:42.388135 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983551d8-25a9-4349-a49e-4832ec248c39 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.403125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983551d8-25a9-4349-a49e-4832ec248c39 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.410163 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f297ecc-e7d1-41b8-b77a-04f14136f290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.411305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f297ecc-e7d1-41b8-b77a-04f14136f290 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.411787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f297ecc-e7d1-41b8-b77a-04f14136f290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.412323 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f297ecc-e7d1-41b8-b77a-04f14136f290 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.413699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f297ecc-e7d1-41b8-b77a-04f14136f290 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:09:42.414265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f297ecc-e7d1-41b8-b77a-04f14136f290 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.416472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 788/1567] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:09:42 2022] POST /identity/v3/projects => generated 369 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:42.417904 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704b28d-106b-477b-aaa5-04d8f16f25a4 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.422608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f297ecc-e7d1-41b8-b77a-04f14136f290 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.454370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 780/1568] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:09:42.454370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f297ecc-e7d1-41b8-b77a-04f14136f290 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.463445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f297ecc-e7d1-41b8-b77a-04f14136f290 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.540293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704b28d-106b-477b-aaa5-04d8f16f25a4 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.548018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca1657-06ea-4952-9c6b-5c6409918304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.549042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca1657-06ea-4952-9c6b-5c6409918304 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.549436 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca1657-06ea-4952-9c6b-5c6409918304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.549809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca1657-06ea-4952-9c6b-5c6409918304 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.550850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca1657-06ea-4952-9c6b-5c6409918304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:09:42.551256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a017c0-1b4b-46f7-83ef-c510e3001cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.551575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca1657-06ea-4952-9c6b-5c6409918304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.551878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a017c0-1b4b-46f7-83ef-c510e3001cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.552339 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a017c0-1b4b-46f7-83ef-c510e3001cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.552624 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a017c0-1b4b-46f7-83ef-c510e3001cf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.553326 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a017c0-1b4b-46f7-83ef-c510e3001cf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.553540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca1657-06ea-4952-9c6b-5c6409918304 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.555282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a017c0-1b4b-46f7-83ef-c510e3001cf5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.582389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 789/1569] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:09:42.582389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca1657-06ea-4952-9c6b-5c6409918304 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.598940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca1657-06ea-4952-9c6b-5c6409918304 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.607843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 781/1570] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:09:42 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:42.607843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a017c0-1b4b-46f7-83ef-c510e3001cf5 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.610603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d4a62-2c64-4474-b9cd-9c91525b1e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.611945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d4a62-2c64-4474-b9cd-9c91525b1e86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.612516 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d4a62-2c64-4474-b9cd-9c91525b1e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.613053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d4a62-2c64-4474-b9cd-9c91525b1e86 admin admin] PATH_INFO: `/v3/projects/e39f08f9bf594c02bf393bb0e5ae12c9/users/50f193cbc4fa49feb19aab6c69144ead/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.640433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d4a62-2c64-4474-b9cd-9c91525b1e86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e39f08f9bf594c02bf393bb0e5ae12c9, user_id=50f193cbc4fa49feb19aab6c69144ead, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.643898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d4a62-2c64-4474-b9cd-9c91525b1e86 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.665930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 790/1571] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:42 2022] PUT /identity/v3/projects/e39f08f9bf594c02bf393bb0e5ae12c9/users/50f193cbc4fa49feb19aab6c69144ead/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:42.672151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d4a62-2c64-4474-b9cd-9c91525b1e86 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.719389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a017c0-1b4b-46f7-83ef-c510e3001cf5 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.760342 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d4a62-2c64-4474-b9cd-9c91525b1e86 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.765718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d89454-191b-421c-aab8-73a594b2d6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.766629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d89454-191b-421c-aab8-73a594b2d6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.767038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d89454-191b-421c-aab8-73a594b2d6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.767468 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d89454-191b-421c-aab8-73a594b2d6d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.768472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d89454-191b-421c-aab8-73a594b2d6d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.770726 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d89454-191b-421c-aab8-73a594b2d6d1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.787092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:42.800644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 791/1572] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:09:42 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:42.802163 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d89454-191b-421c-aab8-73a594b2d6d1 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.804795 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996efa54-0768-4eb4-922b-ca5f08ad3da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.806161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996efa54-0768-4eb4-922b-ca5f08ad3da3 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.806730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996efa54-0768-4eb4-922b-ca5f08ad3da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.807173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996efa54-0768-4eb4-922b-ca5f08ad3da3 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.808127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996efa54-0768-4eb4-922b-ca5f08ad3da3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.811236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996efa54-0768-4eb4-922b-ca5f08ad3da3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.818148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d89454-191b-421c-aab8-73a594b2d6d1 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.826324 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3469c4-48d2-41d4-b825-7fb463c53e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.827325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3469c4-48d2-41d4-b825-7fb463c53e67 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.827742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3469c4-48d2-41d4-b825-7fb463c53e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.828206 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3469c4-48d2-41d4-b825-7fb463c53e67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.829292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3469c4-48d2-41d4-b825-7fb463c53e67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:09:42.829665 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3469c4-48d2-41d4-b825-7fb463c53e67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.832044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3469c4-48d2-41d4-b825-7fb463c53e67 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.834788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:42.838231 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 782/1573] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:09:42 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:42.839105 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996efa54-0768-4eb4-922b-ca5f08ad3da3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.851711 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996efa54-0768-4eb4-922b-ca5f08ad3da3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.856205 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 792/1574] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:09:42.858228 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ba576-5020-43cd-aa2e-f07a75417b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.858936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3469c4-48d2-41d4-b825-7fb463c53e67 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.859151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ba576-5020-43cd-aa2e-f07a75417b3c admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.859302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ba576-5020-43cd-aa2e-f07a75417b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.859652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ba576-5020-43cd-aa2e-f07a75417b3c admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.860489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ba576-5020-43cd-aa2e-f07a75417b3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.864441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ba576-5020-43cd-aa2e-f07a75417b3c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.881831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3469c4-48d2-41d4-b825-7fb463c53e67 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.889227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a2f5b-aaba-4f34-9af2-901f66b560e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.889994 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a2f5b-aaba-4f34-9af2-901f66b560e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.890410 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a2f5b-aaba-4f34-9af2-901f66b560e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.890989 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8a2f5b-aaba-4f34-9af2-901f66b560e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.891816 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a2f5b-aaba-4f34-9af2-901f66b560e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.894003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 783/1575] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:09:42 2022] POST /identity/v3/users => generated 285 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:42.894241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8a2f5b-aaba-4f34-9af2-901f66b560e0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.894868 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ba576-5020-43cd-aa2e-f07a75417b3c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.914771 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ba576-5020-43cd-aa2e-f07a75417b3c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:42.921825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1c429-2add-4a20-9e2c-bb41b75a1929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:42.922656 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1c429-2add-4a20-9e2c-bb41b75a1929 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.923099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1c429-2add-4a20-9e2c-bb41b75a1929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.923474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1c429-2add-4a20-9e2c-bb41b75a1929 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:42.924644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1c429-2add-4a20-9e2c-bb41b75a1929 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:42.927347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1c429-2add-4a20-9e2c-bb41b75a1929 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:42.941302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 793/1576] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:09:42 2022] POST /identity/v3/projects => generated 393 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:42.941302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a2f5b-aaba-4f34-9af2-901f66b560e0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:42.964374 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 784/1577] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:09:42 2022] POST /identity/v3/projects => generated 387 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:42.967714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17991a-db85-405b-a9c2-ac01eaf723be None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:42.968233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17991a-db85-405b-a9c2-ac01eaf723be None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:42.968698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f17991a-db85-405b-a9c2-ac01eaf723be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.003298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5f17991a-db85-405b-a9c2-ac01eaf723be None None] MFA Rules not processed for user `50f193cbc4fa49feb19aab6c69144ead`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:43.005059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5f17991a-db85-405b-a9c2-ac01eaf723be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:43.031426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8a2f5b-aaba-4f34-9af2-901f66b560e0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.087793 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 785/1578] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:42 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:43.088999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17991a-db85-405b-a9c2-ac01eaf723be None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.103619 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1280f73-da10-4f33-b4b8-f4b81a3e5274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.104386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1280f73-da10-4f33-b4b8-f4b81a3e5274 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.104685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1280f73-da10-4f33-b4b8-f4b81a3e5274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.104933 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1280f73-da10-4f33-b4b8-f4b81a3e5274 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.105665 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1280f73-da10-4f33-b4b8-f4b81a3e5274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.109354 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1280f73-da10-4f33-b4b8-f4b81a3e5274 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.144870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 794/1579] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:09:42 2022] POST /identity/v3/users => generated 290 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:43.144870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1280f73-da10-4f33-b4b8-f4b81a3e5274 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.168063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1280f73-da10-4f33-b4b8-f4b81a3e5274 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.177912 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae9db2-b419-4357-9a26-dc062e289bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.178793 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae9db2-b419-4357-9a26-dc062e289bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.180955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae9db2-b419-4357-9a26-dc062e289bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.180955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae9db2-b419-4357-9a26-dc062e289bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.180955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae9db2-b419-4357-9a26-dc062e289bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.182243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae9db2-b419-4357-9a26-dc062e289bf9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.184364 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f17991a-db85-405b-a9c2-ac01eaf723be None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.192104 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec07d1-ab2d-492d-a95d-d15acc0c130a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.192907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec07d1-ab2d-492d-a95d-d15acc0c130a admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.193389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec07d1-ab2d-492d-a95d-d15acc0c130a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.193590 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec07d1-ab2d-492d-a95d-d15acc0c130a admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.194384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec07d1-ab2d-492d-a95d-d15acc0c130a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.195739 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 795/1580] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:43.197394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec07d1-ab2d-492d-a95d-d15acc0c130a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.200308 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae9db2-b419-4357-9a26-dc062e289bf9 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.210562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae9db2-b419-4357-9a26-dc062e289bf9 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.216427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941c9cd-4a3e-43d0-9742-de099fb3d5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.217388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941c9cd-4a3e-43d0-9742-de099fb3d5d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.217675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941c9cd-4a3e-43d0-9742-de099fb3d5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.217956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941c9cd-4a3e-43d0-9742-de099fb3d5d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.222592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 786/1581] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:43.222883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941c9cd-4a3e-43d0-9742-de099fb3d5d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.223076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec07d1-ab2d-492d-a95d-d15acc0c130a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.223228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941c9cd-4a3e-43d0-9742-de099fb3d5d0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.235518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec07d1-ab2d-492d-a95d-d15acc0c130a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.240813 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87634a9-9842-4098-a4f6-52866986f798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.242026 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87634a9-9842-4098-a4f6-52866986f798 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.242416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87634a9-9842-4098-a4f6-52866986f798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.242756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87634a9-9842-4098-a4f6-52866986f798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.243686 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87634a9-9842-4098-a4f6-52866986f798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.245671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87634a9-9842-4098-a4f6-52866986f798 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.254240 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 796/1582] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:09:43 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:43.255098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941c9cd-4a3e-43d0-9742-de099fb3d5d0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.273778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 787/1583] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:43.273778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87634a9-9842-4098-a4f6-52866986f798 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.290072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87634a9-9842-4098-a4f6-52866986f798 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.313957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:43.317754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:43.362989 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941c9cd-4a3e-43d0-9742-de099fb3d5d0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.368353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405f51e-bd7f-485a-b151-46db823f0da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.369401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405f51e-bd7f-485a-b151-46db823f0da3 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.370185 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405f51e-bd7f-485a-b151-46db823f0da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.374014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405f51e-bd7f-485a-b151-46db823f0da3 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.374861 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405f51e-bd7f-485a-b151-46db823f0da3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.376980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405f51e-bd7f-485a-b151-46db823f0da3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.392927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdc706-8622-4b8c-8d3d-982605e7dda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.394360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdc706-8622-4b8c-8d3d-982605e7dda7 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.395212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdc706-8622-4b8c-8d3d-982605e7dda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.395608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdc706-8622-4b8c-8d3d-982605e7dda7 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.397302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbdc706-8622-4b8c-8d3d-982605e7dda7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.401579 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbdc706-8622-4b8c-8d3d-982605e7dda7 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.410556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 797/1584] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:09:43 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:43.411532 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405f51e-bd7f-485a-b151-46db823f0da3 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.426903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405f51e-bd7f-485a-b151-46db823f0da3 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.436819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b5511-897f-4eb8-aba1-41fe7fd99682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.437585 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b5511-897f-4eb8-aba1-41fe7fd99682 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.437891 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b5511-897f-4eb8-aba1-41fe7fd99682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.438100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 788/1585] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:09:43 2022] POST /identity/v3/users => generated 294 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:43.438280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b5511-897f-4eb8-aba1-41fe7fd99682 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.439012 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdc706-8622-4b8c-8d3d-982605e7dda7 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.439195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b5511-897f-4eb8-aba1-41fe7fd99682 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.443308 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b5511-897f-4eb8-aba1-41fe7fd99682 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.453614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdc706-8622-4b8c-8d3d-982605e7dda7 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.460304 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e930b-98c7-49d1-a693-8cf696815269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.461603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e930b-98c7-49d1-a693-8cf696815269 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.462137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e930b-98c7-49d1-a693-8cf696815269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.462687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e930b-98c7-49d1-a693-8cf696815269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.464805 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e930b-98c7-49d1-a693-8cf696815269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.468886 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e930b-98c7-49d1-a693-8cf696815269 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.485791 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 798/1586] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:09:43 2022] POST /identity/v3/projects => generated 403 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:43.487507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b5511-897f-4eb8-aba1-41fe7fd99682 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.494520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 789/1587] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:43.494520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e930b-98c7-49d1-a693-8cf696815269 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.503608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e930b-98c7-49d1-a693-8cf696815269 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.590676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b5511-897f-4eb8-aba1-41fe7fd99682 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.596365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6b634-c6fd-4105-8b5c-3b219eb560ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.596365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6b634-c6fd-4105-8b5c-3b219eb560ed admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.596365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6b634-c6fd-4105-8b5c-3b219eb560ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.596365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6b634-c6fd-4105-8b5c-3b219eb560ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.596947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6b634-c6fd-4105-8b5c-3b219eb560ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.598128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb6b634-c6fd-4105-8b5c-3b219eb560ed admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.602890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f1354-11fb-434c-8944-ca0808bd979c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.603877 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f1354-11fb-434c-8944-ca0808bd979c admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.604385 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f1354-11fb-434c-8944-ca0808bd979c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.604788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f1354-11fb-434c-8944-ca0808bd979c admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.606202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f1354-11fb-434c-8944-ca0808bd979c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.608756 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f1354-11fb-434c-8944-ca0808bd979c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.617872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 790/1588] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:43.619825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6b634-c6fd-4105-8b5c-3b219eb560ed admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.624775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 799/1589] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:43.625808 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f1354-11fb-434c-8944-ca0808bd979c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.639686 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6b634-c6fd-4105-8b5c-3b219eb560ed admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.644704 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f1354-11fb-434c-8944-ca0808bd979c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.646958 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a118db-1a64-4efe-9cfe-da0fac138ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.647812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a118db-1a64-4efe-9cfe-da0fac138ddc admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.648217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a118db-1a64-4efe-9cfe-da0fac138ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.648570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a118db-1a64-4efe-9cfe-da0fac138ddc admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.649397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a118db-1a64-4efe-9cfe-da0fac138ddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.651552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c57099-2e43-4aa5-8a70-5f26e9aca078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.652061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a118db-1a64-4efe-9cfe-da0fac138ddc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.652672 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c57099-2e43-4aa5-8a70-5f26e9aca078 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.652885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c57099-2e43-4aa5-8a70-5f26e9aca078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.653259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c57099-2e43-4aa5-8a70-5f26e9aca078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.654404 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c57099-2e43-4aa5-8a70-5f26e9aca078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.656208 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c57099-2e43-4aa5-8a70-5f26e9aca078 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.677218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 800/1590] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:43.677218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c57099-2e43-4aa5-8a70-5f26e9aca078 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.682312 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 791/1591] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:09:43 2022] POST /identity/v3/users => generated 289 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:43.683419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a118db-1a64-4efe-9cfe-da0fac138ddc admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.698821 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c57099-2e43-4aa5-8a70-5f26e9aca078 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.700034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a118db-1a64-4efe-9cfe-da0fac138ddc admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.705594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98d5a8-b39e-4eef-a85c-d519963042c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.706164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2f9f1-4ef3-4a61-950d-4e809b5c5849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.706615 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98d5a8-b39e-4eef-a85c-d519963042c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.706849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2f9f1-4ef3-4a61-950d-4e809b5c5849 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.707092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98d5a8-b39e-4eef-a85c-d519963042c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.707276 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2f9f1-4ef3-4a61-950d-4e809b5c5849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.707447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2f9f1-4ef3-4a61-950d-4e809b5c5849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.707614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98d5a8-b39e-4eef-a85c-d519963042c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.708171 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2f9f1-4ef3-4a61-950d-4e809b5c5849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.708604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98d5a8-b39e-4eef-a85c-d519963042c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.709945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2f9f1-4ef3-4a61-950d-4e809b5c5849 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.712486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98d5a8-b39e-4eef-a85c-d519963042c2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.726100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 792/1592] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:43.727094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98d5a8-b39e-4eef-a85c-d519963042c2 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.735771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 801/1593] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:43.737525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2f9f1-4ef3-4a61-950d-4e809b5c5849 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.744124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98d5a8-b39e-4eef-a85c-d519963042c2 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.753671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba004e-1ba2-4031-b8e4-22a712a7fad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.753671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba004e-1ba2-4031-b8e4-22a712a7fad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.753671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba004e-1ba2-4031-b8e4-22a712a7fad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.754365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba004e-1ba2-4031-b8e4-22a712a7fad5 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.754897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba004e-1ba2-4031-b8e4-22a712a7fad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.757177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba004e-1ba2-4031-b8e4-22a712a7fad5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.759317 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2f9f1-4ef3-4a61-950d-4e809b5c5849 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.769877 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3f233-42f8-42e7-85f1-6a9247b56d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.771025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3f233-42f8-42e7-85f1-6a9247b56d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.771638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3f233-42f8-42e7-85f1-6a9247b56d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.772168 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3f233-42f8-42e7-85f1-6a9247b56d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.776110 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3f233-42f8-42e7-85f1-6a9247b56d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.781899 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3f233-42f8-42e7-85f1-6a9247b56d29 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.789973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 793/1594] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:09:43 2022] POST /identity/v3/users => generated 308 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:43.789973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba004e-1ba2-4031-b8e4-22a712a7fad5 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.802252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba004e-1ba2-4031-b8e4-22a712a7fad5 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.809878 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 802/1595] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:43.811465 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3f233-42f8-42e7-85f1-6a9247b56d29 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.812801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fb355-c005-49f8-b816-be9cf27ec29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.814348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fb355-c005-49f8-b816-be9cf27ec29c admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.814899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fb355-c005-49f8-b816-be9cf27ec29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.823042 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fb355-c005-49f8-b816-be9cf27ec29c admin admin] PATH_INFO: `/v3/projects/4e8d358f0c964c969da2621cdb44c578/users/2bf2143846654362ba36e5a4e7de18c1/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.830421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3f233-42f8-42e7-85f1-6a9247b56d29 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:43.838705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd4295-e50a-484c-a868-cb687f25f3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:43.840856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd4295-e50a-484c-a868-cb687f25f3fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:43.841397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd4295-e50a-484c-a868-cb687f25f3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:43.841858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd4295-e50a-484c-a868-cb687f25f3fc admin admin] PATH_INFO: `/v3/projects/8c46e98ae6174c1888cb10582f8a0bf4/users/7b4006c43f7944c083cf323db4df2b3b/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:43.849690 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043fb355-c005-49f8-b816-be9cf27ec29c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8d358f0c964c969da2621cdb44c578, user_id=2bf2143846654362ba36e5a4e7de18c1, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.850785 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-043fb355-c005-49f8-b816-be9cf27ec29c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 09:09:43.850785 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 21 09:09:43.850785 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: Jul 21 09:09:43.853359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043fb355-c005-49f8-b816-be9cf27ec29c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.881409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd4295-e50a-484c-a868-cb687f25f3fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c46e98ae6174c1888cb10582f8a0bf4, user_id=7b4006c43f7944c083cf323db4df2b3b, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:43.884573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd4295-e50a-484c-a868-cb687f25f3fc admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:43.886059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 794/1596] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:43 2022] PUT /identity/v3/projects/4e8d358f0c964c969da2621cdb44c578/users/2bf2143846654362ba36e5a4e7de18c1/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:43.887968 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fb355-c005-49f8-b816-be9cf27ec29c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.930252 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 803/1597] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:43 2022] PUT /identity/v3/projects/8c46e98ae6174c1888cb10582f8a0bf4/users/7b4006c43f7944c083cf323db4df2b3b/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:43.930252 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd4295-e50a-484c-a868-cb687f25f3fc admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:43.974678 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:44.015327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:44.041437 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd4295-e50a-484c-a868-cb687f25f3fc admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:44.041759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fb355-c005-49f8-b816-be9cf27ec29c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:44.049668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741f6ed-88d0-4641-af23-596aabc9d247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:44.050049 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65396f6a-db22-4936-b814-a813479e7541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:44.050678 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741f6ed-88d0-4641-af23-596aabc9d247 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:44.050862 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65396f6a-db22-4936-b814-a813479e7541 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:44.051318 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741f6ed-88d0-4641-af23-596aabc9d247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:44.051493 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741f6ed-88d0-4641-af23-596aabc9d247 admin admin] PATH_INFO: `/v3/projects/a13b838c65d74600ae79b97fc5834a67/users/663fe63e5f40438c8060a51f24b114a1/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:44.051668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65396f6a-db22-4936-b814-a813479e7541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:44.052141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65396f6a-db22-4936-b814-a813479e7541 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:44.053377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65396f6a-db22-4936-b814-a813479e7541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:44.055561 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65396f6a-db22-4936-b814-a813479e7541 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:44.071753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 804/1598] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:44.073734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65396f6a-db22-4936-b814-a813479e7541 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:44.078102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741f6ed-88d0-4641-af23-596aabc9d247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a13b838c65d74600ae79b97fc5834a67, user_id=663fe63e5f40438c8060a51f24b114a1, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:44.081165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741f6ed-88d0-4641-af23-596aabc9d247 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:44.089473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65396f6a-db22-4936-b814-a813479e7541 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:44.098085 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756a6c4-45ca-42b1-b64e-d58c57c3fcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:44.099187 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756a6c4-45ca-42b1-b64e-d58c57c3fcb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:44.099653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756a6c4-45ca-42b1-b64e-d58c57c3fcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:44.100178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756a6c4-45ca-42b1-b64e-d58c57c3fcb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:44.102554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756a6c4-45ca-42b1-b64e-d58c57c3fcb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:44.116234 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 795/1599] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:43 2022] PUT /identity/v3/projects/a13b838c65d74600ae79b97fc5834a67/users/663fe63e5f40438c8060a51f24b114a1/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:44.116561 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756a6c4-45ca-42b1-b64e-d58c57c3fcb9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:44.116722 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741f6ed-88d0-4641-af23-596aabc9d247 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:44.126306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 805/1600] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:44.129862 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756a6c4-45ca-42b1-b64e-d58c57c3fcb9 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:44.146283 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756a6c4-45ca-42b1-b64e-d58c57c3fcb9 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:44.216507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741f6ed-88d0-4641-af23-596aabc9d247 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:44.222437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e0b60-e814-46d1-8399-3aab4928e031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:44.223217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e0b60-e814-46d1-8399-3aab4928e031 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:44.223738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e0b60-e814-46d1-8399-3aab4928e031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:44.224171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e0b60-e814-46d1-8399-3aab4928e031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:44.225382 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e0b60-e814-46d1-8399-3aab4928e031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:44.227885 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e0b60-e814-46d1-8399-3aab4928e031 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:44.250593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc97c9-a0ea-4329-a42c-4a8122c3a42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:44.252563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 796/1601] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:44.252860 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc97c9-a0ea-4329-a42c-4a8122c3a42e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:44.253289 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc97c9-a0ea-4329-a42c-4a8122c3a42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:44.253736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc97c9-a0ea-4329-a42c-4a8122c3a42e admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:44.254078 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e0b60-e814-46d1-8399-3aab4928e031 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:44.255075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc97c9-a0ea-4329-a42c-4a8122c3a42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:44.260239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc97c9-a0ea-4329-a42c-4a8122c3a42e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:44.274197 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e0b60-e814-46d1-8399-3aab4928e031 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:44.284981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1179322-533e-4012-b805-200b753ba786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:44.284981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1179322-533e-4012-b805-200b753ba786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:44.285448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 806/1602] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:44.286042 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1179322-533e-4012-b805-200b753ba786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:44.286623 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1179322-533e-4012-b805-200b753ba786 admin admin] PATH_INFO: `/v3/projects/808a7ff7e5f848f6aa646b09641551c4/users/675378ce4e594727a59049f6ce476ce0/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:44.287099 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cd853-7054-44bc-a56c-b9dbe6de8548 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:44.287593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cd853-7054-44bc-a56c-b9dbe6de8548 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:44.288111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cd853-7054-44bc-a56c-b9dbe6de8548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:44.328770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1179322-533e-4012-b805-200b753ba786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=808a7ff7e5f848f6aa646b09641551c4, user_id=675378ce4e594727a59049f6ce476ce0, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:44.332553 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1179322-533e-4012-b805-200b753ba786 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:44.335350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4c2cd853-7054-44bc-a56c-b9dbe6de8548 None None] MFA Rules not processed for user `2bf2143846654362ba36e5a4e7de18c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:44.335350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4c2cd853-7054-44bc-a56c-b9dbe6de8548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:44.363179 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 797/1603] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:44 2022] PUT /identity/v3/projects/808a7ff7e5f848f6aa646b09641551c4/users/675378ce4e594727a59049f6ce476ce0/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:44.366934 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a4f6c-827b-400c-b19f-bf057b12d1ce None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:44.366934 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a4f6c-827b-400c-b19f-bf057b12d1ce None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:44.366934 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a4f6c-827b-400c-b19f-bf057b12d1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:44.399963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ff9a4f6c-827b-400c-b19f-bf057b12d1ce None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:44.401412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ff9a4f6c-827b-400c-b19f-bf057b12d1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:44.439978 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 807/1604] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:44.441470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cd853-7054-44bc-a56c-b9dbe6de8548 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:44.457237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cd853-7054-44bc-a56c-b9dbe6de8548 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:44.500277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 798/1605] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:44.502023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcf833-6481-499f-aecb-3f60282a0b0d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:44.502575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcf833-6481-499f-aecb-3f60282a0b0d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:44.502984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcf833-6481-499f-aecb-3f60282a0b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:44.542648 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:44.543890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-66fcf833-6481-499f-aecb-3f60282a0b0d None None] MFA Rules not processed for user `663fe63e5f40438c8060a51f24b114a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:44.545743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-66fcf833-6481-499f-aecb-3f60282a0b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:44.554900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:44.561774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1adb076-f5f7-402a-87f6-c4f1e156ab7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:44.562870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1adb076-f5f7-402a-87f6-c4f1e156ab7f admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:44.563336 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1adb076-f5f7-402a-87f6-c4f1e156ab7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:44.563790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1adb076-f5f7-402a-87f6-c4f1e156ab7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:44.565021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1adb076-f5f7-402a-87f6-c4f1e156ab7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:44.568074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1adb076-f5f7-402a-87f6-c4f1e156ab7f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:44.595017 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 808/1606] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:44.596041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1adb076-f5f7-402a-87f6-c4f1e156ab7f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:44.610858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1adb076-f5f7-402a-87f6-c4f1e156ab7f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:44.620361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221d937-9cd1-43da-8306-3d3807f7b08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:44.621357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221d937-9cd1-43da-8306-3d3807f7b08b admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:44.621635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221d937-9cd1-43da-8306-3d3807f7b08b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:44.621911 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221d937-9cd1-43da-8306-3d3807f7b08b admin admin] PATH_INFO: `/v3/projects/11c9eb819852493baadbcc11ecce2914/users/e717fb256de24adb8ee6815338a94e5d/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:44.641937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221d937-9cd1-43da-8306-3d3807f7b08b admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c9eb819852493baadbcc11ecce2914, user_id=e717fb256de24adb8ee6815338a94e5d, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:44.645232 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221d937-9cd1-43da-8306-3d3807f7b08b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:44.664365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 809/1607] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:44 2022] PUT /identity/v3/projects/11c9eb819852493baadbcc11ecce2914/users/e717fb256de24adb8ee6815338a94e5d/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:44.664932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221d937-9cd1-43da-8306-3d3807f7b08b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:44.681022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 799/1608] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:44.681022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcf833-6481-499f-aecb-3f60282a0b0d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:44.696944 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcf833-6481-499f-aecb-3f60282a0b0d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:44.748582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221d937-9cd1-43da-8306-3d3807f7b08b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:44.756055 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ae9458-4759-4229-ab52-3cb67be21ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:44.757446 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae9458-4759-4229-ab52-3cb67be21ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:44.757830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae9458-4759-4229-ab52-3cb67be21ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:44.758191 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae9458-4759-4229-ab52-3cb67be21ff4 admin admin] PATH_INFO: `/v3/projects/7efd9ffb80ea497b93cc2e499bda86f7/users/be1372c20ad94145b775663a9037d55a/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:44.782574 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae9458-4759-4229-ab52-3cb67be21ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7efd9ffb80ea497b93cc2e499bda86f7, user_id=be1372c20ad94145b775663a9037d55a, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:44.786131 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ae9458-4759-4229-ab52-3cb67be21ff4 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:44.792377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8dc5b7-4c8d-4abc-baca-f519f2118ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:44.794452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dc5b7-4c8d-4abc-baca-f519f2118ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:44.794955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dc5b7-4c8d-4abc-baca-f519f2118ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:44.795378 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8dc5b7-4c8d-4abc-baca-f519f2118ccb admin admin] PATH_INFO: `/v3/projects/a6ff603c86de4534aac5011389fb7904/users/ebdf979b16f44c42a8295a169d36bf45/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:44.806196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 810/1609] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:44 2022] PUT /identity/v3/projects/7efd9ffb80ea497b93cc2e499bda86f7/users/be1372c20ad94145b775663a9037d55a/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:44.808547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe7f7a-47f8-469d-838d-3d36ebf2ef4a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:44.809331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe7f7a-47f8-469d-838d-3d36ebf2ef4a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:44.809841 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe7f7a-47f8-469d-838d-3d36ebf2ef4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:44.817348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8dc5b7-4c8d-4abc-baca-f519f2118ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ff603c86de4534aac5011389fb7904, user_id=ebdf979b16f44c42a8295a169d36bf45, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:44.820577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8dc5b7-4c8d-4abc-baca-f519f2118ccb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:44.832897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-cdfe7f7a-47f8-469d-838d-3d36ebf2ef4a None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:44.834534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cdfe7f7a-47f8-469d-838d-3d36ebf2ef4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:44.840410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 800/1610] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:44 2022] PUT /identity/v3/projects/a6ff603c86de4534aac5011389fb7904/users/ebdf979b16f44c42a8295a169d36bf45/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:44.840410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8dc5b7-4c8d-4abc-baca-f519f2118ccb admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:44.940189 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8dc5b7-4c8d-4abc-baca-f519f2118ccb admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:44.941031 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 811/1611] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:44.942068 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe7f7a-47f8-469d-838d-3d36ebf2ef4a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:44.952900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50795abd-a2af-485c-bb88-a88b6fcd5ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:44.953943 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50795abd-a2af-485c-bb88-a88b6fcd5ffc admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:44.954534 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50795abd-a2af-485c-bb88-a88b6fcd5ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:44.955021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50795abd-a2af-485c-bb88-a88b6fcd5ffc admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:44.956081 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50795abd-a2af-485c-bb88-a88b6fcd5ffc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:44.958676 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50795abd-a2af-485c-bb88-a88b6fcd5ffc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:44.998976 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 801/1612] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:09:44 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:44.998976 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50795abd-a2af-485c-bb88-a88b6fcd5ffc admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:45.019754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:45.020962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:45.027112 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe7f7a-47f8-469d-838d-3d36ebf2ef4a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:45.088954 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50795abd-a2af-485c-bb88-a88b6fcd5ffc admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:45.097101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0ba9a-49f4-4b7b-b78e-ae7f93eea283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:45.098203 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0ba9a-49f4-4b7b-b78e-ae7f93eea283 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.098635 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0ba9a-49f4-4b7b-b78e-ae7f93eea283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.099093 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0ba9a-49f4-4b7b-b78e-ae7f93eea283 admin admin] PATH_INFO: `/v3/projects/b47c6d295e05423a9e5ad6f0499cbf2a/users/6c62db4b38434cfe886108a72e4c42aa/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.104185 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e9154-1dd0-4ed1-a1fe-c66a5da996c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:45.105044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e9154-1dd0-4ed1-a1fe-c66a5da996c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.105606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e9154-1dd0-4ed1-a1fe-c66a5da996c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.106039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e9154-1dd0-4ed1-a1fe-c66a5da996c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.118466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e9154-1dd0-4ed1-a1fe-c66a5da996c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:45.120697 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e9154-1dd0-4ed1-a1fe-c66a5da996c7 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:45.122715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0ba9a-49f4-4b7b-b78e-ae7f93eea283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b47c6d295e05423a9e5ad6f0499cbf2a, user_id=6c62db4b38434cfe886108a72e4c42aa, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:45.125486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0ba9a-49f4-4b7b-b78e-ae7f93eea283 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:45.133498 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 812/1613] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:09:45.135701 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0a96c-8127-42cf-8976-c8af93b0a235 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.136232 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0a96c-8127-42cf-8976-c8af93b0a235 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.137771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0a96c-8127-42cf-8976-c8af93b0a235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.156614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 802/1614] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:44 2022] PUT /identity/v3/projects/b47c6d295e05423a9e5ad6f0499cbf2a/users/6c62db4b38434cfe886108a72e4c42aa/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:45.156614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0ba9a-49f4-4b7b-b78e-ae7f93eea283 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:45.165630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1cc0a96c-8127-42cf-8976-c8af93b0a235 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:45.167712 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1cc0a96c-8127-42cf-8976-c8af93b0a235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:45.260864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0ba9a-49f4-4b7b-b78e-ae7f93eea283 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:45.268363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8881f1e-6fc8-4264-84e8-6d0b98a18553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:45.269529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8881f1e-6fc8-4264-84e8-6d0b98a18553 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.270096 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8881f1e-6fc8-4264-84e8-6d0b98a18553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.270521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8881f1e-6fc8-4264-84e8-6d0b98a18553 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.271879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8881f1e-6fc8-4264-84e8-6d0b98a18553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:45.275404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8881f1e-6fc8-4264-84e8-6d0b98a18553 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:45.311103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 813/1615] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:45.311103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74907084-7455-4d43-98f3-5ae3f9f853cd None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.311103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74907084-7455-4d43-98f3-5ae3f9f853cd None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.312338 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74907084-7455-4d43-98f3-5ae3f9f853cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.323064 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 803/1616] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:09:45 2022] POST /identity/v3/projects => generated 369 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:45.324643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8881f1e-6fc8-4264-84e8-6d0b98a18553 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:45.345515 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-74907084-7455-4d43-98f3-5ae3f9f853cd None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:45.347393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-74907084-7455-4d43-98f3-5ae3f9f853cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:45.451537 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 814/1617] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:45.452786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74907084-7455-4d43-98f3-5ae3f9f853cd None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:45.491037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8881f1e-6fc8-4264-84e8-6d0b98a18553 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:45.502147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a2a45-a0b0-4750-ba0a-665e5b46fe9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:45.503743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a2a45-a0b0-4750-ba0a-665e5b46fe9e admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.504233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a2a45-a0b0-4750-ba0a-665e5b46fe9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.505609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a2a45-a0b0-4750-ba0a-665e5b46fe9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.507372 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a2a45-a0b0-4750-ba0a-665e5b46fe9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:45.510443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a2a45-a0b0-4750-ba0a-665e5b46fe9e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:45.533140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 804/1618] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:45.535014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a2a45-a0b0-4750-ba0a-665e5b46fe9e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:45.555856 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a2a45-a0b0-4750-ba0a-665e5b46fe9e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:45.562042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:45.569218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10c8fd-e6c5-42c4-997c-8d1631f35ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:45.570515 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10c8fd-e6c5-42c4-997c-8d1631f35ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.570904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10c8fd-e6c5-42c4-997c-8d1631f35ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.571404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10c8fd-e6c5-42c4-997c-8d1631f35ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.571756 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74907084-7455-4d43-98f3-5ae3f9f853cd None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:45.573300 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10c8fd-e6c5-42c4-997c-8d1631f35ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:45.577896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10c8fd-e6c5-42c4-997c-8d1631f35ae4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:45.581410 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14a3e0-ca9d-40c1-9013-733af40209f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:45.582722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14a3e0-ca9d-40c1-9013-733af40209f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.583339 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14a3e0-ca9d-40c1-9013-733af40209f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.583885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14a3e0-ca9d-40c1-9013-733af40209f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.598013 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14a3e0-ca9d-40c1-9013-733af40209f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:45.599773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14a3e0-ca9d-40c1-9013-733af40209f7 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:45.611448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 805/1619] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:09:45 2022] POST /identity/v3/users => generated 293 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:45.612955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 815/1620] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:09:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:09:45.613998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14a3e0-ca9d-40c1-9013-733af40209f7 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:45.615358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f4da9-6695-43f2-8404-62bc23744421 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.616047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f4da9-6695-43f2-8404-62bc23744421 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.616351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f4da9-6695-43f2-8404-62bc23744421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.623538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:45.625806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14a3e0-ca9d-40c1-9013-733af40209f7 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:45.630258 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1744c-d236-4d78-a4f0-9e7f6639d12d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:45.630947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1744c-d236-4d78-a4f0-9e7f6639d12d service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.631397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1744c-d236-4d78-a4f0-9e7f6639d12d service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.631684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1744c-d236-4d78-a4f0-9e7f6639d12d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.643338 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1744c-d236-4d78-a4f0-9e7f6639d12d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:45.644909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1744c-d236-4d78-a4f0-9e7f6639d12d service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:45.657699 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 816/1621] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:09:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:09:45.657931 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-081f4da9-6695-43f2-8404-62bc23744421 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:45.658339 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1744c-d236-4d78-a4f0-9e7f6639d12d service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:45.660140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-081f4da9-6695-43f2-8404-62bc23744421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:45.669925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1744c-d236-4d78-a4f0-9e7f6639d12d service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:45.674779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdf86d-dc45-426c-91c4-c7b0107c05b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:45.678742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 806/1622] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:45.678950 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdf86d-dc45-426c-91c4-c7b0107c05b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.678950 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdf86d-dc45-426c-91c4-c7b0107c05b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.678950 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdf86d-dc45-426c-91c4-c7b0107c05b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.679181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f4da9-6695-43f2-8404-62bc23744421 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:45.679373 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdf86d-dc45-426c-91c4-c7b0107c05b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:45.679485 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdf86d-dc45-426c-91c4-c7b0107c05b8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:45.692453 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f4da9-6695-43f2-8404-62bc23744421 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:45.697121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafb883-6e4c-4805-8720-c26770256008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:45.697812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafb883-6e4c-4805-8720-c26770256008 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.698088 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafb883-6e4c-4805-8720-c26770256008 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.698341 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafb883-6e4c-4805-8720-c26770256008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.716617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 817/1623] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:09:45 2022] POST /identity/v3/users => generated 291 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:45.716617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdf86d-dc45-426c-91c4-c7b0107c05b8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:45.723728 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafb883-6e4c-4805-8720-c26770256008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:45.726397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafb883-6e4c-4805-8720-c26770256008 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:45.729898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdf86d-dc45-426c-91c4-c7b0107c05b8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:45.739885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ead8a-1d9e-40ac-bdd7-a4d314c56aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:45.741455 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ead8a-1d9e-40ac-bdd7-a4d314c56aba admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.741897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ead8a-1d9e-40ac-bdd7-a4d314c56aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.742370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ead8a-1d9e-40ac-bdd7-a4d314c56aba admin admin] PATH_INFO: `/v3/projects/a6ff603c86de4534aac5011389fb7904/users/ebdf979b16f44c42a8295a169d36bf45/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.745528 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 807/1624] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:09:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:09:45.747798 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafb883-6e4c-4805-8720-c26770256008 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:45.758062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ead8a-1d9e-40ac-bdd7-a4d314c56aba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ff603c86de4534aac5011389fb7904, user_id=ebdf979b16f44c42a8295a169d36bf45, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:45.769328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafb883-6e4c-4805-8720-c26770256008 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:45.770197 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ead8a-1d9e-40ac-bdd7-a4d314c56aba admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:45.804436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154b215-bc2c-4779-bed7-86936d25082f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:45.805048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154b215-bc2c-4779-bed7-86936d25082f admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.805343 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154b215-bc2c-4779-bed7-86936d25082f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.805629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154b215-bc2c-4779-bed7-86936d25082f admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.806564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154b215-bc2c-4779-bed7-86936d25082f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:45.808402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154b215-bc2c-4779-bed7-86936d25082f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:45.810085 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 818/1625] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:45 2022] PUT /identity/v3/projects/a6ff603c86de4534aac5011389fb7904/users/ebdf979b16f44c42a8295a169d36bf45/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:45.812181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ead8a-1d9e-40ac-bdd7-a4d314c56aba admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:45.838753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 808/1626] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:45.840211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154b215-bc2c-4779-bed7-86936d25082f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:45.870220 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154b215-bc2c-4779-bed7-86936d25082f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:45.922637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ead8a-1d9e-40ac-bdd7-a4d314c56aba admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:45.932129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd134ce2-b980-4414-b03a-d1bb83647c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:45.933292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd134ce2-b980-4414-b03a-d1bb83647c6e service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.934018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd134ce2-b980-4414-b03a-d1bb83647c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.934842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd134ce2-b980-4414-b03a-d1bb83647c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.987134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb41e1-cfac-470d-8de6-0c3c324618ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:45.987965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb41e1-cfac-470d-8de6-0c3c324618ac admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:45.988420 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb41e1-cfac-470d-8de6-0c3c324618ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:45.988891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb41e1-cfac-470d-8de6-0c3c324618ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:45.989947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb41e1-cfac-470d-8de6-0c3c324618ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:45.992033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb41e1-cfac-470d-8de6-0c3c324618ac admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:46.010567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 809/1627] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:46.013938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841d045-b291-40e8-843d-524fa09aab27 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:46.014427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841d045-b291-40e8-843d-524fa09aab27 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:46.014807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841d045-b291-40e8-843d-524fa09aab27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:46.056123 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a841d045-b291-40e8-843d-524fa09aab27 None None] MFA Rules not processed for user `ebdf979b16f44c42a8295a169d36bf45`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:46.057942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a841d045-b291-40e8-843d-524fa09aab27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:46.145326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd134ce2-b980-4414-b03a-d1bb83647c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:46.152326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd134ce2-b980-4414-b03a-d1bb83647c6e service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:46.179109 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 810/1628] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:46 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:46.180718 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841d045-b291-40e8-843d-524fa09aab27 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:46.182696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 819/1629] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:09:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:09:46.185358 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd134ce2-b980-4414-b03a-d1bb83647c6e service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:46.197186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841d045-b291-40e8-843d-524fa09aab27 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:46.206370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f215c-2c47-4000-9efd-dc1cb9f3c816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:46.208069 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f215c-2c47-4000-9efd-dc1cb9f3c816 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:46.208362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f215c-2c47-4000-9efd-dc1cb9f3c816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:46.208638 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f215c-2c47-4000-9efd-dc1cb9f3c816 admin admin] PATH_INFO: `/v3/projects/c7d7caff274e410b88030564f5721b55/users/7b7aa1adab4044349f22d1af8897fae1/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:46.216265 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd134ce2-b980-4414-b03a-d1bb83647c6e service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:46.228406 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b81d3a-a3b2-4cf9-9c93-f7aae7744426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:46.231385 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b81d3a-a3b2-4cf9-9c93-f7aae7744426 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:46.232779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b81d3a-a3b2-4cf9-9c93-f7aae7744426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:46.233949 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f215c-2c47-4000-9efd-dc1cb9f3c816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7d7caff274e410b88030564f5721b55, user_id=7b7aa1adab4044349f22d1af8897fae1, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:46.234479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b81d3a-a3b2-4cf9-9c93-f7aae7744426 admin admin] PATH_INFO: `/v3/projects/afd3a52cd4264669a504c822b8b0f6b2/users/02e13ae3a506410a8fad5bac4d1ebfa7/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:46.237672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f215c-2c47-4000-9efd-dc1cb9f3c816 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:46.269302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 811/1630] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:46 2022] PUT /identity/v3/projects/c7d7caff274e410b88030564f5721b55/users/7b7aa1adab4044349f22d1af8897fae1/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:46.269302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f215c-2c47-4000-9efd-dc1cb9f3c816 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:46.278896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b81d3a-a3b2-4cf9-9c93-f7aae7744426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afd3a52cd4264669a504c822b8b0f6b2, user_id=02e13ae3a506410a8fad5bac4d1ebfa7, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:46.285735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b81d3a-a3b2-4cf9-9c93-f7aae7744426 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:46.320371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:46.328453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 820/1631] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:46 2022] PUT /identity/v3/projects/afd3a52cd4264669a504c822b8b0f6b2/users/02e13ae3a506410a8fad5bac4d1ebfa7/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:46.328453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b81d3a-a3b2-4cf9-9c93-f7aae7744426 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:46.365535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f215c-2c47-4000-9efd-dc1cb9f3c816 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:46.431624 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:46.455546 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8c2ac-5fa2-4c15-a4be-636f671501d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:46.456649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8c2ac-5fa2-4c15-a4be-636f671501d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:46.456649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8c2ac-5fa2-4c15-a4be-636f671501d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:46.457028 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8c2ac-5fa2-4c15-a4be-636f671501d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:46.511282 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b81d3a-a3b2-4cf9-9c93-f7aae7744426 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:46.523736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e5442-4e6f-402a-99d9-26a78e3b9c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:46.525580 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e5442-4e6f-402a-99d9-26a78e3b9c6c admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:46.526313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e5442-4e6f-402a-99d9-26a78e3b9c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:46.527148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e5442-4e6f-402a-99d9-26a78e3b9c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:46.529856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e5442-4e6f-402a-99d9-26a78e3b9c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:46.534175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e5442-4e6f-402a-99d9-26a78e3b9c6c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:46.554791 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8c2ac-5fa2-4c15-a4be-636f671501d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:46.556634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8c2ac-5fa2-4c15-a4be-636f671501d2 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:46.566045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 821/1632] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:46.566045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e5442-4e6f-402a-99d9-26a78e3b9c6c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:46.582649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 812/1633] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:09:46 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:09:46.586096 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8c2ac-5fa2-4c15-a4be-636f671501d2 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:46.591061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e5442-4e6f-402a-99d9-26a78e3b9c6c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:46.602148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311830b-cbd3-4870-8abb-c7d09c985ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:46.603046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8c2ac-5fa2-4c15-a4be-636f671501d2 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:46.603438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311830b-cbd3-4870-8abb-c7d09c985ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:46.604093 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311830b-cbd3-4870-8abb-c7d09c985ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:46.604736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311830b-cbd3-4870-8abb-c7d09c985ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:46.606957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311830b-cbd3-4870-8abb-c7d09c985ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:46.610659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311830b-cbd3-4870-8abb-c7d09c985ee7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:46.619493 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c51847-f927-4c7e-9e46-751c1ac809cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:46.620731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c51847-f927-4c7e-9e46-751c1ac809cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:46.621675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c51847-f927-4c7e-9e46-751c1ac809cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:46.623236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c51847-f927-4c7e-9e46-751c1ac809cd admin admin] PATH_INFO: `/v3/projects/c7d7caff274e410b88030564f5721b55/users/7b7aa1adab4044349f22d1af8897fae1/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:46.626044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c51847-f927-4c7e-9e46-751c1ac809cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7d7caff274e410b88030564f5721b55, user_id=7b7aa1adab4044349f22d1af8897fae1, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:46.634060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c51847-f927-4c7e-9e46-751c1ac809cd admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:46.666319 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 822/1635] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:46.666625 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 813/1635] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:46 2022] PUT /identity/v3/projects/c7d7caff274e410b88030564f5721b55/users/7b7aa1adab4044349f22d1af8897fae1/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:46.666815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311830b-cbd3-4870-8abb-c7d09c985ee7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:46.670086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6503d1d-9553-44d7-8d21-4b9a2a1b6b0e None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:46.670580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6503d1d-9553-44d7-8d21-4b9a2a1b6b0e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:46.671077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6503d1d-9553-44d7-8d21-4b9a2a1b6b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:46.681209 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311830b-cbd3-4870-8abb-c7d09c985ee7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:46.732166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d6503d1d-9553-44d7-8d21-4b9a2a1b6b0e None None] MFA Rules not processed for user `7b7aa1adab4044349f22d1af8897fae1`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:46.736071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d6503d1d-9553-44d7-8d21-4b9a2a1b6b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:46.835024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257bab6-96f3-47bc-8453-e8ce8949283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:46.835024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257bab6-96f3-47bc-8453-e8ce8949283a admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:46.836472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257bab6-96f3-47bc-8453-e8ce8949283a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:46.836472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257bab6-96f3-47bc-8453-e8ce8949283a admin admin] PATH_INFO: `/v3/projects/afd3a52cd4264669a504c822b8b0f6b2/users/02e13ae3a506410a8fad5bac4d1ebfa7/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:46.846851 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257bab6-96f3-47bc-8453-e8ce8949283a admin admin] RBAC: Authorizing `identity:create_grant(project_id=afd3a52cd4264669a504c822b8b0f6b2, user_id=02e13ae3a506410a8fad5bac4d1ebfa7, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:46.857047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f257bab6-96f3-47bc-8453-e8ce8949283a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:46.912261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 823/1636] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:46 2022] PUT /identity/v3/projects/afd3a52cd4264669a504c822b8b0f6b2/users/02e13ae3a506410a8fad5bac4d1ebfa7/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:46.923562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7153ac-c42a-420e-a408-81ff0db779b9 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:46.924096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7153ac-c42a-420e-a408-81ff0db779b9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:46.924841 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7153ac-c42a-420e-a408-81ff0db779b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:46.972535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 814/1637] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:47.000525 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6503d1d-9553-44d7-8d21-4b9a2a1b6b0e None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:47.023606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3c7153ac-c42a-420e-a408-81ff0db779b9 None None] MFA Rules not processed for user `02e13ae3a506410a8fad5bac4d1ebfa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:47.031146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3c7153ac-c42a-420e-a408-81ff0db779b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:47.046873 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:47.166352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6503d1d-9553-44d7-8d21-4b9a2a1b6b0e None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:47.197683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 824/1638] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:46 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:47.200032 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7153ac-c42a-420e-a408-81ff0db779b9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:47.372465 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:47.385086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e600d-bfa8-4cdc-955d-52d9fa778d23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:47.389014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e600d-bfa8-4cdc-955d-52d9fa778d23 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:47.391252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e600d-bfa8-4cdc-955d-52d9fa778d23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:47.393676 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e600d-bfa8-4cdc-955d-52d9fa778d23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:47.464249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7153ac-c42a-420e-a408-81ff0db779b9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:47.477734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2f6ec-4969-433b-8835-757d061a8749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:47.478965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2f6ec-4969-433b-8835-757d061a8749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:47.479424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2f6ec-4969-433b-8835-757d061a8749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:47.480096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2f6ec-4969-433b-8835-757d061a8749 admin admin] PATH_INFO: `/v3/users/50f193cbc4fa49feb19aab6c69144ead` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:47.485367 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2f6ec-4969-433b-8835-757d061a8749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50f193cbc4fa49feb19aab6c69144ead)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:47.489830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2f6ec-4969-433b-8835-757d061a8749 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:47.591571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e600d-bfa8-4cdc-955d-52d9fa778d23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:47.598801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e600d-bfa8-4cdc-955d-52d9fa778d23 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:47.626703 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 815/1639] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:09:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:09:47.627555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e600d-bfa8-4cdc-955d-52d9fa778d23 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:47.646625 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e600d-bfa8-4cdc-955d-52d9fa778d23 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:47.648648 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-56f2f6ec-4969-433b-8835-757d061a8749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50f193cbc4fa49feb19aab6c69144ead'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:47.650602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-56f2f6ec-4969-433b-8835-757d061a8749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50f193cbc4fa49feb19aab6c69144ead'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:47.661680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a52217-debb-4af5-bfbf-4bc9f036e405 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:47.664164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a52217-debb-4af5-bfbf-4bc9f036e405 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:47.664164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a52217-debb-4af5-bfbf-4bc9f036e405 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:47.664607 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a52217-debb-4af5-bfbf-4bc9f036e405 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:47.668455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a1a52217-debb-4af5-bfbf-4bc9f036e405 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:47.672439 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-56f2f6ec-4969-433b-8835-757d061a8749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50f193cbc4fa49feb19aab6c69144ead'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:47.707874 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a52217-debb-4af5-bfbf-4bc9f036e405 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:47.709169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a52217-debb-4af5-bfbf-4bc9f036e405 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:47.711064 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-56f2f6ec-4969-433b-8835-757d061a8749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50f193cbc4fa49feb19aab6c69144ead'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:47.764311 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 816/1640] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:09:47 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:09:47.769363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-56f2f6ec-4969-433b-8835-757d061a8749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50f193cbc4fa49feb19aab6c69144ead'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:47.807737 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 825/1641] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:09:47 2022] DELETE /identity/v3/users/50f193cbc4fa49feb19aab6c69144ead => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:47.816501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81caf6-cbab-4b70-a4fc-a9cfcb925861 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:47.817180 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81caf6-cbab-4b70-a4fc-a9cfcb925861 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:47.818030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81caf6-cbab-4b70-a4fc-a9cfcb925861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:47.873911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6c81caf6-cbab-4b70-a4fc-a9cfcb925861 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:47.880032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6c81caf6-cbab-4b70-a4fc-a9cfcb925861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:48.138300 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:48.166590 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 817/1642] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:09:48.288341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2f6ec-4969-433b-8835-757d061a8749 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:48.291168 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-56f2f6ec-4969-433b-8835-757d061a8749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:48.487520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2f6ec-4969-433b-8835-757d061a8749 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:48.499971 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5109f7a0-1208-496f-8239-36f6ac66a9c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:48.501259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109f7a0-1208-496f-8239-36f6ac66a9c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:48.501855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109f7a0-1208-496f-8239-36f6ac66a9c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:48.502367 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109f7a0-1208-496f-8239-36f6ac66a9c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:48.535462 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5109f7a0-1208-496f-8239-36f6ac66a9c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:48.543667 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5109f7a0-1208-496f-8239-36f6ac66a9c5 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:48.577904 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 826/1643] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:09:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:09:48.728408 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81caf6-cbab-4b70-a4fc-a9cfcb925861 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:48.731182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6c81caf6-cbab-4b70-a4fc-a9cfcb925861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:48.754369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81caf6-cbab-4b70-a4fc-a9cfcb925861 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:48.760846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd736300-3dd9-45d3-83b0-6971b7479891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:48.762632 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd736300-3dd9-45d3-83b0-6971b7479891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:48.763063 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd736300-3dd9-45d3-83b0-6971b7479891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:48.763495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd736300-3dd9-45d3-83b0-6971b7479891 admin admin] PATH_INFO: `/v3/users/2bf2143846654362ba36e5a4e7de18c1` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:48.765285 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd736300-3dd9-45d3-83b0-6971b7479891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf2143846654362ba36e5a4e7de18c1)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:48.766585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-fd736300-3dd9-45d3-83b0-6971b7479891 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 09:09:48.766585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 21 09:09:48.766585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: Jul 21 09:09:48.768240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd736300-3dd9-45d3-83b0-6971b7479891 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:48.976665 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fd736300-3dd9-45d3-83b0-6971b7479891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf2143846654362ba36e5a4e7de18c1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:49.004705 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fd736300-3dd9-45d3-83b0-6971b7479891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf2143846654362ba36e5a4e7de18c1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:49.024717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fd736300-3dd9-45d3-83b0-6971b7479891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf2143846654362ba36e5a4e7de18c1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:49.065950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fd736300-3dd9-45d3-83b0-6971b7479891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf2143846654362ba36e5a4e7de18c1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:49.067780 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fd736300-3dd9-45d3-83b0-6971b7479891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf2143846654362ba36e5a4e7de18c1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:49.091961 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 818/1644] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:09:48 2022] DELETE /identity/v3/users/2bf2143846654362ba36e5a4e7de18c1 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:49.095606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634d5aa-b359-4fa8-9397-3e7f1d4115c3 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:49.096952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634d5aa-b359-4fa8-9397-3e7f1d4115c3 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:49.101587 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634d5aa-b359-4fa8-9397-3e7f1d4115c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:49.130574 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3634d5aa-b359-4fa8-9397-3e7f1d4115c3 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:49.132479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3634d5aa-b359-4fa8-9397-3e7f1d4115c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:49.274595 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:49.329627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 827/1645] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:49.476098 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd736300-3dd9-45d3-83b0-6971b7479891 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:49.478382 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fd736300-3dd9-45d3-83b0-6971b7479891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:49.829280 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd736300-3dd9-45d3-83b0-6971b7479891 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:49.841826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d1f0c-7d31-48d4-97fe-eb2ee9a27226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:49.846357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d1f0c-7d31-48d4-97fe-eb2ee9a27226 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:49.847753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d1f0c-7d31-48d4-97fe-eb2ee9a27226 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:49.849345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d1f0c-7d31-48d4-97fe-eb2ee9a27226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:49.898263 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d1f0c-7d31-48d4-97fe-eb2ee9a27226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:49.905595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d1f0c-7d31-48d4-97fe-eb2ee9a27226 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:49.939025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 819/1646] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:09:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:09:50.719819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634d5aa-b359-4fa8-9397-3e7f1d4115c3 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:50.723672 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3634d5aa-b359-4fa8-9397-3e7f1d4115c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:50.765218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634d5aa-b359-4fa8-9397-3e7f1d4115c3 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:50.788861 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:50.790768 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:50.791541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:50.793048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] PATH_INFO: `/v3/projects/e39f08f9bf594c02bf393bb0e5ae12c9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:50.796801 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e39f08f9bf594c02bf393bb0e5ae12c9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:50.800175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: WARNING py.warnings [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 09:09:50.800175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: warnings.warn(msg) Jul 21 09:09:50.800175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: Jul 21 09:09:50.825531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:51.048285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e39f08f9bf594c02bf393bb0e5ae12c9', 'request_id': 'req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:51.080797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e39f08f9bf594c02bf393bb0e5ae12c9', 'request_id': 'req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:51.153079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e39f08f9bf594c02bf393bb0e5ae12c9', 'request_id': 'req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:51.195254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] The token cache is being invalidate because project e39f08f9bf594c02bf393bb0e5ae12c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:09:51.195785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:51.201129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:51.208901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 828/1647] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:09:50 2022] DELETE /identity/v3/projects/e39f08f9bf594c02bf393bb0e5ae12c9 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:51.212886 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d1f0c-7d31-48d4-97fe-eb2ee9a27226 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:51.242315 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d1f0c-7d31-48d4-97fe-eb2ee9a27226 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:51.419683 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624331d7-a612-48b5-bca3-e9442ca28357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:51.422286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624331d7-a612-48b5-bca3-e9442ca28357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:51.422286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624331d7-a612-48b5-bca3-e9442ca28357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:51.422905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624331d7-a612-48b5-bca3-e9442ca28357 admin admin] PATH_INFO: `/v3/users/ebdf979b16f44c42a8295a169d36bf45` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:51.425369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624331d7-a612-48b5-bca3-e9442ca28357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebdf979b16f44c42a8295a169d36bf45)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:51.429855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624331d7-a612-48b5-bca3-e9442ca28357 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:51.473960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:51.538780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:51.544030 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:51.613112 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a4b6b-ed71-4d74-8ff0-18a31cb75407 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:51.615637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-624331d7-a612-48b5-bca3-e9442ca28357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebdf979b16f44c42a8295a169d36bf45'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:51.646228 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-624331d7-a612-48b5-bca3-e9442ca28357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebdf979b16f44c42a8295a169d36bf45'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:51.655504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5174ce3-7ac8-406f-be33-54b81763cb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:51.659954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5174ce3-7ac8-406f-be33-54b81763cb7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:51.665284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5174ce3-7ac8-406f-be33-54b81763cb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:51.666526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5174ce3-7ac8-406f-be33-54b81763cb7b admin admin] PATH_INFO: `/v3/users/663fe63e5f40438c8060a51f24b114a1` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:51.678558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5174ce3-7ac8-406f-be33-54b81763cb7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=663fe63e5f40438c8060a51f24b114a1)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:51.682056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5174ce3-7ac8-406f-be33-54b81763cb7b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:51.685236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-624331d7-a612-48b5-bca3-e9442ca28357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebdf979b16f44c42a8295a169d36bf45'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:51.722171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-624331d7-a612-48b5-bca3-e9442ca28357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebdf979b16f44c42a8295a169d36bf45'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:51.723631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-624331d7-a612-48b5-bca3-e9442ca28357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebdf979b16f44c42a8295a169d36bf45'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:51.741913 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 820/1648] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:09:51 2022] DELETE /identity/v3/users/ebdf979b16f44c42a8295a169d36bf45 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:09:51.888399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b5174ce3-7ac8-406f-be33-54b81763cb7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '663fe63e5f40438c8060a51f24b114a1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:51.889329 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b5174ce3-7ac8-406f-be33-54b81763cb7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '663fe63e5f40438c8060a51f24b114a1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:51.905951 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b5174ce3-7ac8-406f-be33-54b81763cb7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '663fe63e5f40438c8060a51f24b114a1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:51.957894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b5174ce3-7ac8-406f-be33-54b81763cb7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '663fe63e5f40438c8060a51f24b114a1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:51.985482 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b5174ce3-7ac8-406f-be33-54b81763cb7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '663fe63e5f40438c8060a51f24b114a1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:52.007674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 829/1649] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:09:51 2022] DELETE /identity/v3/users/663fe63e5f40438c8060a51f24b114a1 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:52.020420 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c61ae-b6a7-4afb-b827-e88827fbd40f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:52.020932 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c61ae-b6a7-4afb-b827-e88827fbd40f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:52.021527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c61ae-b6a7-4afb-b827-e88827fbd40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:52.097617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-418c61ae-b6a7-4afb-b827-e88827fbd40f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:52.099205 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-418c61ae-b6a7-4afb-b827-e88827fbd40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:52.288677 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 821/1650] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:52.359890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5174ce3-7ac8-406f-be33-54b81763cb7b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:52.368131 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b5174ce3-7ac8-406f-be33-54b81763cb7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:52.544237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5174ce3-7ac8-406f-be33-54b81763cb7b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:52.554350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf3ce9-b3e4-4856-b8e1-082911b10b14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:52.555174 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf3ce9-b3e4-4856-b8e1-082911b10b14 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:52.555469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf3ce9-b3e4-4856-b8e1-082911b10b14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:52.555750 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf3ce9-b3e4-4856-b8e1-082911b10b14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:52.575514 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf3ce9-b3e4-4856-b8e1-082911b10b14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:52.577424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf3ce9-b3e4-4856-b8e1-082911b10b14 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:52.584453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:52.592468 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 830/1651] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:09:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:09:52.746955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c61ae-b6a7-4afb-b827-e88827fbd40f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:52.749571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-418c61ae-b6a7-4afb-b827-e88827fbd40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:52.780565 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c61ae-b6a7-4afb-b827-e88827fbd40f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:52.789621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d359c-f52b-46f9-a7e9-97a59af35706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:52.791231 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d359c-f52b-46f9-a7e9-97a59af35706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:52.791752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d359c-f52b-46f9-a7e9-97a59af35706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:52.793241 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d359c-f52b-46f9-a7e9-97a59af35706 admin admin] PATH_INFO: `/v3/projects/4e8d358f0c964c969da2621cdb44c578` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:52.795333 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d359c-f52b-46f9-a7e9-97a59af35706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e8d358f0c964c969da2621cdb44c578)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:52.799864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d359c-f52b-46f9-a7e9-97a59af35706 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:52.915831 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-356d359c-f52b-46f9-a7e9-97a59af35706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e8d358f0c964c969da2621cdb44c578', 'request_id': 'req-356d359c-f52b-46f9-a7e9-97a59af35706'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:52.938862 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-356d359c-f52b-46f9-a7e9-97a59af35706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e8d358f0c964c969da2621cdb44c578', 'request_id': 'req-356d359c-f52b-46f9-a7e9-97a59af35706'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:52.958407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-356d359c-f52b-46f9-a7e9-97a59af35706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e8d358f0c964c969da2621cdb44c578', 'request_id': 'req-356d359c-f52b-46f9-a7e9-97a59af35706'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:52.989567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-356d359c-f52b-46f9-a7e9-97a59af35706 admin admin] The token cache is being invalidate because project 4e8d358f0c964c969da2621cdb44c578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:09:52.990350 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-356d359c-f52b-46f9-a7e9-97a59af35706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:52.992728 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-356d359c-f52b-46f9-a7e9-97a59af35706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:52.997359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 822/1652] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:09:52 2022] DELETE /identity/v3/projects/4e8d358f0c964c969da2621cdb44c578 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:53.004000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf3ce9-b3e4-4856-b8e1-082911b10b14 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:53.018551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf3ce9-b3e4-4856-b8e1-082911b10b14 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:53.111114 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c3f88-dda0-48eb-b3c7-0039e9c825c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:53.112392 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c3f88-dda0-48eb-b3c7-0039e9c825c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:53.112698 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c3f88-dda0-48eb-b3c7-0039e9c825c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:53.113042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c3f88-dda0-48eb-b3c7-0039e9c825c9 admin admin] PATH_INFO: `/v3/users/7b7aa1adab4044349f22d1af8897fae1` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:53.114478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c3f88-dda0-48eb-b3c7-0039e9c825c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b7aa1adab4044349f22d1af8897fae1)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:53.116650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c3f88-dda0-48eb-b3c7-0039e9c825c9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:53.205346 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bb5c3f88-dda0-48eb-b3c7-0039e9c825c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b7aa1adab4044349f22d1af8897fae1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:53.206166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bb5c3f88-dda0-48eb-b3c7-0039e9c825c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b7aa1adab4044349f22d1af8897fae1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:53.220171 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bb5c3f88-dda0-48eb-b3c7-0039e9c825c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b7aa1adab4044349f22d1af8897fae1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:53.249024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bb5c3f88-dda0-48eb-b3c7-0039e9c825c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b7aa1adab4044349f22d1af8897fae1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:53.262301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bb5c3f88-dda0-48eb-b3c7-0039e9c825c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b7aa1adab4044349f22d1af8897fae1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:53.286982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 831/1653] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:09:53 2022] DELETE /identity/v3/users/7b7aa1adab4044349f22d1af8897fae1 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:09:54.372946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d359c-f52b-46f9-a7e9-97a59af35706 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:54.375604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-356d359c-f52b-46f9-a7e9-97a59af35706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:54.400425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d359c-f52b-46f9-a7e9-97a59af35706 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:54.441163 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:54.548802 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64afd6-1449-4d49-9349-c65623ec56a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:54.554249 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64afd6-1449-4d49-9349-c65623ec56a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:54.555457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64afd6-1449-4d49-9349-c65623ec56a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:54.557589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64afd6-1449-4d49-9349-c65623ec56a6 admin admin] PATH_INFO: `/v3/projects/c7d7caff274e410b88030564f5721b55` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:54.563033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b64afd6-1449-4d49-9349-c65623ec56a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7d7caff274e410b88030564f5721b55)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:54.568852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b64afd6-1449-4d49-9349-c65623ec56a6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:54.661255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c3f88-dda0-48eb-b3c7-0039e9c825c9 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:54.663469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-bb5c3f88-dda0-48eb-b3c7-0039e9c825c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:54.708631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c3f88-dda0-48eb-b3c7-0039e9c825c9 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:54.709715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4b64afd6-1449-4d49-9349-c65623ec56a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7d7caff274e410b88030564f5721b55', 'request_id': 'req-4b64afd6-1449-4d49-9349-c65623ec56a6'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:54.738493 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4b64afd6-1449-4d49-9349-c65623ec56a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7d7caff274e410b88030564f5721b55', 'request_id': 'req-4b64afd6-1449-4d49-9349-c65623ec56a6'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:54.761224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4b64afd6-1449-4d49-9349-c65623ec56a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7d7caff274e410b88030564f5721b55', 'request_id': 'req-4b64afd6-1449-4d49-9349-c65623ec56a6'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:54.810765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4b64afd6-1449-4d49-9349-c65623ec56a6 admin admin] The token cache is being invalidate because project c7d7caff274e410b88030564f5721b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:09:54.811094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4b64afd6-1449-4d49-9349-c65623ec56a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:54.812155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4b64afd6-1449-4d49-9349-c65623ec56a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:54.822242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 823/1654] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:09:54 2022] DELETE /identity/v3/projects/c7d7caff274e410b88030564f5721b55 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:54.825875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b9f0c-aba9-4154-bdd0-fb137460b797 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:54.826167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b9f0c-aba9-4154-bdd0-fb137460b797 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:54.828513 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b9f0c-aba9-4154-bdd0-fb137460b797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:54.887852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a86b9f0c-aba9-4154-bdd0-fb137460b797 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:54.889401 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a86b9f0c-aba9-4154-bdd0-fb137460b797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:54.935510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17526e-be4e-4fef-ae82-10a200713dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:54.937519 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17526e-be4e-4fef-ae82-10a200713dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:54.937816 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17526e-be4e-4fef-ae82-10a200713dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:54.938101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17526e-be4e-4fef-ae82-10a200713dc7 admin admin] PATH_INFO: `/v3/projects/a6ff603c86de4534aac5011389fb7904` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:54.940086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17526e-be4e-4fef-ae82-10a200713dc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ff603c86de4534aac5011389fb7904)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:54.943463 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17526e-be4e-4fef-ae82-10a200713dc7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:54.993608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:55.087816 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fb17526e-be4e-4fef-ae82-10a200713dc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ff603c86de4534aac5011389fb7904', 'request_id': 'req-fb17526e-be4e-4fef-ae82-10a200713dc7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:55.094847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 824/1655] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:55.108978 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b9f0c-aba9-4154-bdd0-fb137460b797 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:55.113695 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a86b9f0c-aba9-4154-bdd0-fb137460b797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:55.117863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fb17526e-be4e-4fef-ae82-10a200713dc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ff603c86de4534aac5011389fb7904', 'request_id': 'req-fb17526e-be4e-4fef-ae82-10a200713dc7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:55.145951 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b9f0c-aba9-4154-bdd0-fb137460b797 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:55.150868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fb17526e-be4e-4fef-ae82-10a200713dc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ff603c86de4534aac5011389fb7904', 'request_id': 'req-fb17526e-be4e-4fef-ae82-10a200713dc7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:55.213291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fb17526e-be4e-4fef-ae82-10a200713dc7 admin admin] The token cache is being invalidate because project a6ff603c86de4534aac5011389fb7904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:09:55.213534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fb17526e-be4e-4fef-ae82-10a200713dc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:55.218827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fb17526e-be4e-4fef-ae82-10a200713dc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:55.227781 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 832/1656] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:09:54 2022] DELETE /identity/v3/projects/a6ff603c86de4534aac5011389fb7904 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:55.229522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966c2dc-d8e8-4cd6-9b54-14e1c30e1a58 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:55.230538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966c2dc-d8e8-4cd6-9b54-14e1c30e1a58 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:55.230538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966c2dc-d8e8-4cd6-9b54-14e1c30e1a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:55.348444 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-5966c2dc-d8e8-4cd6-9b54-14e1c30e1a58 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:55.362429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:55.369908 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5966c2dc-d8e8-4cd6-9b54-14e1c30e1a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:55.404935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc119f-edb3-4825-ba97-e42c06448c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:55.406319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc119f-edb3-4825-ba97-e42c06448c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:55.406611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc119f-edb3-4825-ba97-e42c06448c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:55.407045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc119f-edb3-4825-ba97-e42c06448c2b admin admin] PATH_INFO: `/v3/projects/a13b838c65d74600ae79b97fc5834a67` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:55.415867 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc119f-edb3-4825-ba97-e42c06448c2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a13b838c65d74600ae79b97fc5834a67)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:55.418808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc119f-edb3-4825-ba97-e42c06448c2b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:55.546049 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0fdc119f-edb3-4825-ba97-e42c06448c2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a13b838c65d74600ae79b97fc5834a67', 'request_id': 'req-0fdc119f-edb3-4825-ba97-e42c06448c2b'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:55.565584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 833/1657] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:55.570985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5486a-7bf6-4528-a346-9d8b4b5eff24 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:55.571620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5486a-7bf6-4528-a346-9d8b4b5eff24 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:55.573423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5486a-7bf6-4528-a346-9d8b4b5eff24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:55.575993 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0fdc119f-edb3-4825-ba97-e42c06448c2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a13b838c65d74600ae79b97fc5834a67', 'request_id': 'req-0fdc119f-edb3-4825-ba97-e42c06448c2b'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:55.601600 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0fdc119f-edb3-4825-ba97-e42c06448c2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a13b838c65d74600ae79b97fc5834a67', 'request_id': 'req-0fdc119f-edb3-4825-ba97-e42c06448c2b'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:55.636139 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0fdc119f-edb3-4825-ba97-e42c06448c2b admin admin] The token cache is being invalidate because project a13b838c65d74600ae79b97fc5834a67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:09:55.637527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0fdc119f-edb3-4825-ba97-e42c06448c2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:55.637672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0fdc119f-edb3-4825-ba97-e42c06448c2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:55.642478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 825/1658] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:09:55 2022] DELETE /identity/v3/projects/a13b838c65d74600ae79b97fc5834a67 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:55.643740 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f2d5486a-7bf6-4528-a346-9d8b4b5eff24 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:55.644851 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc119f-edb3-4825-ba97-e42c06448c2b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:55.645858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f2d5486a-7bf6-4528-a346-9d8b4b5eff24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:55.646009 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0fdc119f-edb3-4825-ba97-e42c06448c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:55.857090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 834/1659] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:55.861613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5486a-7bf6-4528-a346-9d8b4b5eff24 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:55.863307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc119f-edb3-4825-ba97-e42c06448c2b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:55.878390 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f2d5486a-7bf6-4528-a346-9d8b4b5eff24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:55.884866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8dd6c4-d3ce-429e-b473-6308b1a2c84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:55.885618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dd6c4-d3ce-429e-b473-6308b1a2c84b admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:55.885923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dd6c4-d3ce-429e-b473-6308b1a2c84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:55.886197 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dd6c4-d3ce-429e-b473-6308b1a2c84b admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:55.887493 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8dd6c4-d3ce-429e-b473-6308b1a2c84b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:09:55.887795 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8dd6c4-d3ce-429e-b473-6308b1a2c84b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:55.889506 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8dd6c4-d3ce-429e-b473-6308b1a2c84b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:55.933037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 826/1660] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:09:55.934951 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df166f5a-4327-4965-992b-e878d34810f0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:55.935241 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df166f5a-4327-4965-992b-e878d34810f0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:55.935519 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df166f5a-4327-4965-992b-e878d34810f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:55.945242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5486a-7bf6-4528-a346-9d8b4b5eff24 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:55.971089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121ec75-246c-4b76-8f09-d9e201670eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:55.974955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-df166f5a-4327-4965-992b-e878d34810f0 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:55.976272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-df166f5a-4327-4965-992b-e878d34810f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:55.978769 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121ec75-246c-4b76-8f09-d9e201670eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:55.979269 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121ec75-246c-4b76-8f09-d9e201670eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:55.979762 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121ec75-246c-4b76-8f09-d9e201670eac admin admin] PATH_INFO: `/v3/users/02e13ae3a506410a8fad5bac4d1ebfa7` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:55.984151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121ec75-246c-4b76-8f09-d9e201670eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=02e13ae3a506410a8fad5bac4d1ebfa7)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:56.000810 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121ec75-246c-4b76-8f09-d9e201670eac admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:56.002791 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 827/1661] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:56.005821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df166f5a-4327-4965-992b-e878d34810f0 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:56.034502 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df166f5a-4327-4965-992b-e878d34810f0 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:56.040273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e66148-e3d6-46b3-9812-f0ed8137fab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:56.040963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e66148-e3d6-46b3-9812-f0ed8137fab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:56.041233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e66148-e3d6-46b3-9812-f0ed8137fab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:56.041578 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e66148-e3d6-46b3-9812-f0ed8137fab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:56.042333 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e66148-e3d6-46b3-9812-f0ed8137fab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:56.044914 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e66148-e3d6-46b3-9812-f0ed8137fab5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:56.102443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 828/1662] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:09:56 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:56.103351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e66148-e3d6-46b3-9812-f0ed8137fab5 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:56.205242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:56.212696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:56.216105 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c121ec75-246c-4b76-8f09-d9e201670eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02e13ae3a506410a8fad5bac4d1ebfa7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:56.220080 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c121ec75-246c-4b76-8f09-d9e201670eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02e13ae3a506410a8fad5bac4d1ebfa7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:56.241533 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c121ec75-246c-4b76-8f09-d9e201670eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02e13ae3a506410a8fad5bac4d1ebfa7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:56.260076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e66148-e3d6-46b3-9812-f0ed8137fab5 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:56.291282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73998f6d-ca69-4d7b-b540-8b0672287fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:56.292102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73998f6d-ca69-4d7b-b540-8b0672287fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:56.292243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73998f6d-ca69-4d7b-b540-8b0672287fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:56.292654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73998f6d-ca69-4d7b-b540-8b0672287fb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:56.293574 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73998f6d-ca69-4d7b-b540-8b0672287fb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:09:56.293885 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73998f6d-ca69-4d7b-b540-8b0672287fb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:56.300131 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73998f6d-ca69-4d7b-b540-8b0672287fb0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:56.316770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c121ec75-246c-4b76-8f09-d9e201670eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02e13ae3a506410a8fad5bac4d1ebfa7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:56.324296 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 829/1663] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:09:56.325305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73998f6d-ca69-4d7b-b540-8b0672287fb0 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:56.337733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-73998f6d-ca69-4d7b-b540-8b0672287fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:56.350442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c121ec75-246c-4b76-8f09-d9e201670eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02e13ae3a506410a8fad5bac4d1ebfa7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:09:56.388178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 835/1664] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:09:55 2022] DELETE /identity/v3/users/02e13ae3a506410a8fad5bac4d1ebfa7 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:56.388178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121ec75-246c-4b76-8f09-d9e201670eac admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:56.397429 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c121ec75-246c-4b76-8f09-d9e201670eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:56.402821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73998f6d-ca69-4d7b-b540-8b0672287fb0 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:56.510513 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bf367-db25-4e39-9795-013e9cf133b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:56.513289 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bf367-db25-4e39-9795-013e9cf133b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:56.514563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bf367-db25-4e39-9795-013e9cf133b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:56.515081 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bf367-db25-4e39-9795-013e9cf133b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:56.516083 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095bf367-db25-4e39-9795-013e9cf133b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:56.518594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095bf367-db25-4e39-9795-013e9cf133b3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:56.573166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 830/1665] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:09:56 2022] POST /identity/v3/users => generated 283 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:56.578541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bf367-db25-4e39-9795-013e9cf133b3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:56.579041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121ec75-246c-4b76-8f09-d9e201670eac admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:56.592104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af5d22-9d5b-42d6-b96c-ec74320eb8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:56.592801 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af5d22-9d5b-42d6-b96c-ec74320eb8ba admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:56.593085 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af5d22-9d5b-42d6-b96c-ec74320eb8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:56.593362 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af5d22-9d5b-42d6-b96c-ec74320eb8ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:56.594097 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af5d22-9d5b-42d6-b96c-ec74320eb8ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:56.597951 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af5d22-9d5b-42d6-b96c-ec74320eb8ba admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:56.658283 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bf367-db25-4e39-9795-013e9cf133b3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:56.688680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d37bd3-f802-46e8-afcd-1d6bc7e23029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:56.690329 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 836/1666] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:09:56 2022] POST /identity/v3/projects => generated 381 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:56.690954 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d37bd3-f802-46e8-afcd-1d6bc7e23029 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:56.697278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d37bd3-f802-46e8-afcd-1d6bc7e23029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:56.698074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d37bd3-f802-46e8-afcd-1d6bc7e23029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:56.701298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af5d22-9d5b-42d6-b96c-ec74320eb8ba admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:56.702399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d37bd3-f802-46e8-afcd-1d6bc7e23029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:56.710412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d37bd3-f802-46e8-afcd-1d6bc7e23029 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:56.748831 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 831/1667] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:56.753314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d37bd3-f802-46e8-afcd-1d6bc7e23029 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:56.784473 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d37bd3-f802-46e8-afcd-1d6bc7e23029 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:56.851803 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af5d22-9d5b-42d6-b96c-ec74320eb8ba admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:56.858869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c0007-8ee4-4211-b612-af392ca9a031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:56.860419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c0007-8ee4-4211-b612-af392ca9a031 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:56.860876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c0007-8ee4-4211-b612-af392ca9a031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:56.861304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c0007-8ee4-4211-b612-af392ca9a031 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:56.862179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c0007-8ee4-4211-b612-af392ca9a031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:56.866107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c0007-8ee4-4211-b612-af392ca9a031 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:56.963109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 837/1668] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:09:56 2022] POST /identity/v3/users => generated 291 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:56.970933 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c0007-8ee4-4211-b612-af392ca9a031 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:56.997061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8145764-12ac-43b3-90ca-b6e08d817827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:56.999262 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8145764-12ac-43b3-90ca-b6e08d817827 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:56.999812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8145764-12ac-43b3-90ca-b6e08d817827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:57.000307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8145764-12ac-43b3-90ca-b6e08d817827 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:57.001670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8145764-12ac-43b3-90ca-b6e08d817827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:57.002657 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c0007-8ee4-4211-b612-af392ca9a031 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:57.006590 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8145764-12ac-43b3-90ca-b6e08d817827 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:57.014778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64142fa-b888-46f9-8973-d4ba9edc6b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:57.016828 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64142fa-b888-46f9-8973-d4ba9edc6b5b admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:57.017142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64142fa-b888-46f9-8973-d4ba9edc6b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:57.017450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64142fa-b888-46f9-8973-d4ba9edc6b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:57.018258 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64142fa-b888-46f9-8973-d4ba9edc6b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:57.021795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64142fa-b888-46f9-8973-d4ba9edc6b5b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:57.044506 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 838/1669] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:57.057426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 832/1670] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:57.059538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8145764-12ac-43b3-90ca-b6e08d817827 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:57.065603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64142fa-b888-46f9-8973-d4ba9edc6b5b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:57.085273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8145764-12ac-43b3-90ca-b6e08d817827 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:57.095150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64142fa-b888-46f9-8973-d4ba9edc6b5b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:57.097651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462016b-19c0-4688-8c03-69c0df6603eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:57.098566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462016b-19c0-4688-8c03-69c0df6603eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:57.098846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462016b-19c0-4688-8c03-69c0df6603eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:57.099122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462016b-19c0-4688-8c03-69c0df6603eb admin admin] PATH_INFO: `/v3/projects/25cd385fd699436b954f9b5c38353386/users/d39267e4092b487bb2e51656e34e46dc/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:57.101405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fcc5dd-46a8-4440-a105-2ad7f2d29257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:57.103964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcc5dd-46a8-4440-a105-2ad7f2d29257 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:57.106156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcc5dd-46a8-4440-a105-2ad7f2d29257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:57.106380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcc5dd-46a8-4440-a105-2ad7f2d29257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:57.109942 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fcc5dd-46a8-4440-a105-2ad7f2d29257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:57.112713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fcc5dd-46a8-4440-a105-2ad7f2d29257 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:57.147791 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462016b-19c0-4688-8c03-69c0df6603eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=25cd385fd699436b954f9b5c38353386, user_id=d39267e4092b487bb2e51656e34e46dc, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:57.149776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 839/1671] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:57.155048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fcc5dd-46a8-4440-a105-2ad7f2d29257 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:57.159237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462016b-19c0-4688-8c03-69c0df6603eb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:57.172210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fcc5dd-46a8-4440-a105-2ad7f2d29257 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:57.191099 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c5f33-fc2f-4131-a6d5-3c701108beff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:57.192527 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c5f33-fc2f-4131-a6d5-3c701108beff admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:57.192896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c5f33-fc2f-4131-a6d5-3c701108beff admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:57.193249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c5f33-fc2f-4131-a6d5-3c701108beff admin admin] PATH_INFO: `/v3/projects/d21db19a8b044bd4a224e99976d0e386/users/d3857f0ca1f74afe9c42532d2fcabf9b/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:57.201615 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 833/1672] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:57 2022] PUT /identity/v3/projects/25cd385fd699436b954f9b5c38353386/users/d39267e4092b487bb2e51656e34e46dc/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:57.209279 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a4cbd-fdf6-4d3e-b982-8e0d63bf2faa None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:57.209571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a4cbd-fdf6-4d3e-b982-8e0d63bf2faa None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:57.209841 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:57.210024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a4cbd-fdf6-4d3e-b982-8e0d63bf2faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:57.229285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c5f33-fc2f-4131-a6d5-3c701108beff admin admin] RBAC: Authorizing `identity:create_grant(project_id=d21db19a8b044bd4a224e99976d0e386, user_id=d3857f0ca1f74afe9c42532d2fcabf9b, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:57.241992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c5f33-fc2f-4131-a6d5-3c701108beff admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:57.263273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-386a4cbd-fdf6-4d3e-b982-8e0d63bf2faa None None] MFA Rules not processed for user `d39267e4092b487bb2e51656e34e46dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:57.264935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-386a4cbd-fdf6-4d3e-b982-8e0d63bf2faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:57.284063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 840/1673] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:57 2022] PUT /identity/v3/projects/d21db19a8b044bd4a224e99976d0e386/users/d3857f0ca1f74afe9c42532d2fcabf9b/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:57.309814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a9a55-2fe2-49c7-908f-b6f0e27eaff7 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:57.310125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a9a55-2fe2-49c7-908f-b6f0e27eaff7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:57.310417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a9a55-2fe2-49c7-908f-b6f0e27eaff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:57.324901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:57.392193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c38a9a55-2fe2-49c7-908f-b6f0e27eaff7 None None] MFA Rules not processed for user `d3857f0ca1f74afe9c42532d2fcabf9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:57.393529 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c38a9a55-2fe2-49c7-908f-b6f0e27eaff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:57.453170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 834/1674] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:57.455595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a4cbd-fdf6-4d3e-b982-8e0d63bf2faa None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:57.651468 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 841/1675] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:09:57.659472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a9a55-2fe2-49c7-908f-b6f0e27eaff7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:57.684310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a4cbd-fdf6-4d3e-b982-8e0d63bf2faa None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:57.906399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a9a55-2fe2-49c7-908f-b6f0e27eaff7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:57.913107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3ec7a-0e96-4b0e-9d68-c0864d444a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:57.914413 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3ec7a-0e96-4b0e-9d68-c0864d444a83 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:57.915200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3ec7a-0e96-4b0e-9d68-c0864d444a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:57.915594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3ec7a-0e96-4b0e-9d68-c0864d444a83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:57.916826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3ec7a-0e96-4b0e-9d68-c0864d444a83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:57.921362 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3ec7a-0e96-4b0e-9d68-c0864d444a83 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:57.930255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd2dce-4d14-423a-9421-e1fe83f08e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:57.931316 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd2dce-4d14-423a-9421-e1fe83f08e80 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:57.931604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd2dce-4d14-423a-9421-e1fe83f08e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:57.931884 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd2dce-4d14-423a-9421-e1fe83f08e80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:57.932675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd2dce-4d14-423a-9421-e1fe83f08e80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:57.940209 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd2dce-4d14-423a-9421-e1fe83f08e80 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:57.988602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 842/1676] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:09:57 2022] POST /identity/v3/projects => generated 381 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:09:57.991590 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3ec7a-0e96-4b0e-9d68-c0864d444a83 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:58.006149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 835/1677] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:09:57 2022] POST /identity/v3/projects => generated 363 bytes in 552 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:58.013307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd2dce-4d14-423a-9421-e1fe83f08e80 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:58.148921 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:58.180775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd2dce-4d14-423a-9421-e1fe83f08e80 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:58.197349 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb0fea-873c-443e-93ae-e5941b231984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:58.198063 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb0fea-873c-443e-93ae-e5941b231984 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:58.198352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb0fea-873c-443e-93ae-e5941b231984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:58.198631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb0fea-873c-443e-93ae-e5941b231984 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:58.200343 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb0fea-873c-443e-93ae-e5941b231984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:58.202063 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb0fea-873c-443e-93ae-e5941b231984 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:58.262281 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 836/1678] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:09:58 2022] POST /identity/v3/users => generated 288 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:09:58.269610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb0fea-873c-443e-93ae-e5941b231984 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:58.295128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3ec7a-0e96-4b0e-9d68-c0864d444a83 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:58.296441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb0fea-873c-443e-93ae-e5941b231984 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:58.321736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f28c59-5bb0-467e-936d-5e1e96b18c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:58.322978 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f28c59-5bb0-467e-936d-5e1e96b18c29 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:58.323600 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f28c59-5bb0-467e-936d-5e1e96b18c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:58.324815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f28c59-5bb0-467e-936d-5e1e96b18c29 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:58.326468 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f28c59-5bb0-467e-936d-5e1e96b18c29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:58.328521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9f25a-6140-476b-80f8-97ba157787c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:58.331103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9f25a-6140-476b-80f8-97ba157787c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:58.332313 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9f25a-6140-476b-80f8-97ba157787c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:58.332962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9f25a-6140-476b-80f8-97ba157787c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:58.334246 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9f25a-6140-476b-80f8-97ba157787c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:58.335204 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f28c59-5bb0-467e-936d-5e1e96b18c29 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:58.338241 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9f25a-6140-476b-80f8-97ba157787c0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:58.356428 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 837/1679] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:09:58.363800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9f25a-6140-476b-80f8-97ba157787c0 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:58.383387 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9f25a-6140-476b-80f8-97ba157787c0 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:58.383599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:58.392836 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb997f-c8b9-4838-8638-a9dbb18978b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:58.394087 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb997f-c8b9-4838-8638-a9dbb18978b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:58.394775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb997f-c8b9-4838-8638-a9dbb18978b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:58.395300 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb997f-c8b9-4838-8638-a9dbb18978b7 admin admin] PATH_INFO: `/v3/projects/7b6bf14245944a619c58b2619c87471a/users/21a7a758f56448e8a4133818b50fa166/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:58.429207 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb997f-c8b9-4838-8638-a9dbb18978b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6bf14245944a619c58b2619c87471a, user_id=21a7a758f56448e8a4133818b50fa166, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:58.433627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 843/1680] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:09:57 2022] POST /identity/v3/users => generated 297 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:09:58.435131 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb997f-c8b9-4838-8638-a9dbb18978b7 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:58.435512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f28c59-5bb0-467e-936d-5e1e96b18c29 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:58.488281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f28c59-5bb0-467e-936d-5e1e96b18c29 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:58.506429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 838/1681] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:58 2022] PUT /identity/v3/projects/7b6bf14245944a619c58b2619c87471a/users/21a7a758f56448e8a4133818b50fa166/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:58.512761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb997f-c8b9-4838-8638-a9dbb18978b7 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:58.520717 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c97e7-1311-4dc3-a056-8728c88776f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:58.521925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c97e7-1311-4dc3-a056-8728c88776f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:58.522444 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c97e7-1311-4dc3-a056-8728c88776f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:58.522957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c97e7-1311-4dc3-a056-8728c88776f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:58.524428 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c97e7-1311-4dc3-a056-8728c88776f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:58.533470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c97e7-1311-4dc3-a056-8728c88776f7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:58.592709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 844/1682] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:58.594306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c97e7-1311-4dc3-a056-8728c88776f7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:58.626062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c97e7-1311-4dc3-a056-8728c88776f7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:58.803988 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf96348-952b-4606-a054-56fe3b8c73b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:58.804923 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf96348-952b-4606-a054-56fe3b8c73b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:58.805208 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf96348-952b-4606-a054-56fe3b8c73b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:58.805484 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf96348-952b-4606-a054-56fe3b8c73b1 admin admin] PATH_INFO: `/v3/projects/871235f3f2e0474d9f57df38dba4fcf2/users/86bd1581bdae43b9a7e630aeb2937874/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:58.838883 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb997f-c8b9-4838-8638-a9dbb18978b7 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:58.856769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1dccb3-b96e-4385-a2e2-9456188767be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:58.856769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1dccb3-b96e-4385-a2e2-9456188767be admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:58.856769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1dccb3-b96e-4385-a2e2-9456188767be admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:58.856769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1dccb3-b96e-4385-a2e2-9456188767be admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:58.857366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1dccb3-b96e-4385-a2e2-9456188767be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:58.864613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf96348-952b-4606-a054-56fe3b8c73b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=871235f3f2e0474d9f57df38dba4fcf2, user_id=86bd1581bdae43b9a7e630aeb2937874, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:58.868581 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1dccb3-b96e-4385-a2e2-9456188767be admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:58.875339 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf96348-952b-4606-a054-56fe3b8c73b1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:58.891744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 839/1683] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:58.897623 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1dccb3-b96e-4385-a2e2-9456188767be admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:58.919809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 845/1684] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:58 2022] PUT /identity/v3/projects/871235f3f2e0474d9f57df38dba4fcf2/users/86bd1581bdae43b9a7e630aeb2937874/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:58.925139 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf96348-952b-4606-a054-56fe3b8c73b1 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:58.966283 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1dccb3-b96e-4385-a2e2-9456188767be admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:59.275028 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced8ab6-abdc-4021-9876-ab267dfa245a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:59.277427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced8ab6-abdc-4021-9876-ab267dfa245a admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:59.277717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced8ab6-abdc-4021-9876-ab267dfa245a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:59.277999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced8ab6-abdc-4021-9876-ab267dfa245a admin admin] PATH_INFO: `/v3/projects/7b6bf14245944a619c58b2619c87471a/users/21a7a758f56448e8a4133818b50fa166/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:59.287091 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced8ab6-abdc-4021-9876-ab267dfa245a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6bf14245944a619c58b2619c87471a, user_id=21a7a758f56448e8a4133818b50fa166, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:59.291101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf96348-952b-4606-a054-56fe3b8c73b1 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:59.293648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced8ab6-abdc-4021-9876-ab267dfa245a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:59.318371 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc7f88-8ed2-4e99-bc22-44d6dac57d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:59.319516 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc7f88-8ed2-4e99-bc22-44d6dac57d9f admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:59.324703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc7f88-8ed2-4e99-bc22-44d6dac57d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:59.324918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc7f88-8ed2-4e99-bc22-44d6dac57d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:59.326119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc7f88-8ed2-4e99-bc22-44d6dac57d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:59.328762 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 840/1685] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:58 2022] PUT /identity/v3/projects/7b6bf14245944a619c58b2619c87471a/users/21a7a758f56448e8a4133818b50fa166/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:59.329107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc7f88-8ed2-4e99-bc22-44d6dac57d9f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:59.339656 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed446c0-908b-4018-a199-b507bb64d2f1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:59.340029 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed446c0-908b-4018-a199-b507bb64d2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:59.340319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed446c0-908b-4018-a199-b507bb64d2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:59.408293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 846/1686] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 481 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:09:59.411534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc7f88-8ed2-4e99-bc22-44d6dac57d9f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:59.411959 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2ed446c0-908b-4018-a199-b507bb64d2f1 None None] MFA Rules not processed for user `21a7a758f56448e8a4133818b50fa166`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:09:59.414143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2ed446c0-908b-4018-a199-b507bb64d2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:59.460636 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc7f88-8ed2-4e99-bc22-44d6dac57d9f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:09:59.482569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:59.655297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 841/1687] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:09:59 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:09:59.665189 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed446c0-908b-4018-a199-b507bb64d2f1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:59.669244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2ed446c0-908b-4018-a199-b507bb64d2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:59.759043 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c80fc-c48a-45b6-989b-fc2e6e7974a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:09:59.759893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c80fc-c48a-45b6-989b-fc2e6e7974a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:09:59.760202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c80fc-c48a-45b6-989b-fc2e6e7974a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:09:59.760480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c80fc-c48a-45b6-989b-fc2e6e7974a3 admin admin] PATH_INFO: `/v3/projects/871235f3f2e0474d9f57df38dba4fcf2/users/86bd1581bdae43b9a7e630aeb2937874/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:09:59.777037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c80fc-c48a-45b6-989b-fc2e6e7974a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=871235f3f2e0474d9f57df38dba4fcf2, user_id=86bd1581bdae43b9a7e630aeb2937874, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:09:59.779889 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c80fc-c48a-45b6-989b-fc2e6e7974a3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:09:59.802628 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:09:59.830502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 847/1688] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:09:59 2022] PUT /identity/v3/projects/871235f3f2e0474d9f57df38dba4fcf2/users/86bd1581bdae43b9a7e630aeb2937874/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:09:59.831194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c80fc-c48a-45b6-989b-fc2e6e7974a3 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:09:59.837607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-876c80fc-c48a-45b6-989b-fc2e6e7974a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:09:59.917838 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed446c0-908b-4018-a199-b507bb64d2f1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:00.036140 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c80fc-c48a-45b6-989b-fc2e6e7974a3 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:00.049285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9891957-e882-443b-aa99-2221aed4ebd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:00.050363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9891957-e882-443b-aa99-2221aed4ebd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:00.051159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9891957-e882-443b-aa99-2221aed4ebd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:00.051388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9891957-e882-443b-aa99-2221aed4ebd4 admin admin] PATH_INFO: `/v3/projects/afd3a52cd4264669a504c822b8b0f6b2` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:00.055838 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9891957-e882-443b-aa99-2221aed4ebd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afd3a52cd4264669a504c822b8b0f6b2)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:00.058165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9891957-e882-443b-aa99-2221aed4ebd4 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:00.169051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ccea9-0302-493b-a28c-ce81a0b63bd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:00.172415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ccea9-0302-493b-a28c-ce81a0b63bd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:00.173244 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d9891957-e882-443b-aa99-2221aed4ebd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afd3a52cd4264669a504c822b8b0f6b2', 'request_id': 'req-d9891957-e882-443b-aa99-2221aed4ebd4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:00.173774 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ccea9-0302-493b-a28c-ce81a0b63bd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:00.174229 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ccea9-0302-493b-a28c-ce81a0b63bd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:00.183372 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d9891957-e882-443b-aa99-2221aed4ebd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afd3a52cd4264669a504c822b8b0f6b2', 'request_id': 'req-d9891957-e882-443b-aa99-2221aed4ebd4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:00.200466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d9891957-e882-443b-aa99-2221aed4ebd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afd3a52cd4264669a504c822b8b0f6b2', 'request_id': 'req-d9891957-e882-443b-aa99-2221aed4ebd4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:00.232591 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d9891957-e882-443b-aa99-2221aed4ebd4 admin admin] The token cache is being invalidate because project afd3a52cd4264669a504c822b8b0f6b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:00.233537 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d9891957-e882-443b-aa99-2221aed4ebd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:00.234733 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d9891957-e882-443b-aa99-2221aed4ebd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:00.241237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 848/1689] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:09:59 2022] DELETE /identity/v3/projects/afd3a52cd4264669a504c822b8b0f6b2 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:00.244267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93acff0-375b-4976-b43a-1c8d1e3d9af9 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:00.244922 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93acff0-375b-4976-b43a-1c8d1e3d9af9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:00.245316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93acff0-375b-4976-b43a-1c8d1e3d9af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:00.307691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a93acff0-375b-4976-b43a-1c8d1e3d9af9 None None] MFA Rules not processed for user `86bd1581bdae43b9a7e630aeb2937874`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:00.309792 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a93acff0-375b-4976-b43a-1c8d1e3d9af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:00.421157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ccea9-0302-493b-a28c-ce81a0b63bd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:00.425588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ccea9-0302-493b-a28c-ce81a0b63bd3 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:00.457350 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 842/1690] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:09:59 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 792 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:00.461124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba4428-940a-44a9-80d0-9f3994d459c0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:00.461779 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba4428-940a-44a9-80d0-9f3994d459c0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:00.462812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba4428-940a-44a9-80d0-9f3994d459c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:00.520979 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 849/1691] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:00.523005 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150220ec-7091-4868-8b1d-6f11d2e1c70c None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:00.525938 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150220ec-7091-4868-8b1d-6f11d2e1c70c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:00.530187 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150220ec-7091-4868-8b1d-6f11d2e1c70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:00.570904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5aba4428-940a-44a9-80d0-9f3994d459c0 None None] MFA Rules not processed for user `7b4006c43f7944c083cf323db4df2b3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:00.572728 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5aba4428-940a-44a9-80d0-9f3994d459c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:00.608241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-150220ec-7091-4868-8b1d-6f11d2e1c70c None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:00.619199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-150220ec-7091-4868-8b1d-6f11d2e1c70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:00.634839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:00.636738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:00.826960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 850/1692] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:00.826960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150220ec-7091-4868-8b1d-6f11d2e1c70c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:00.826960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-150220ec-7091-4868-8b1d-6f11d2e1c70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:00.860031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 843/1693] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:00.864878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c328ab3-5601-41ad-ac82-12c6a9db11e3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:00.866049 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c328ab3-5601-41ad-ac82-12c6a9db11e3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:00.866480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c328ab3-5601-41ad-ac82-12c6a9db11e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:00.920070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3c328ab3-5601-41ad-ac82-12c6a9db11e3 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:00.922263 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3c328ab3-5601-41ad-ac82-12c6a9db11e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:00.952244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 844/1694] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:00.952881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac89864-d276-47ba-b95b-6dc3e6c9ff5d None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:00.953402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac89864-d276-47ba-b95b-6dc3e6c9ff5d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:00.956494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac89864-d276-47ba-b95b-6dc3e6c9ff5d None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:00.961873 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 845/1695] 10.222.0.13 () {54 vars in 962 bytes} [Thu Jul 21 09:10:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:10:00.963547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac89864-d276-47ba-b95b-6dc3e6c9ff5d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:00.994978 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac89864-d276-47ba-b95b-6dc3e6c9ff5d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:01.007493 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c00aff-99c9-41c6-b442-258e027e05b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:01.008474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c00aff-99c9-41c6-b442-258e027e05b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:01.009037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c00aff-99c9-41c6-b442-258e027e05b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:01.010569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c00aff-99c9-41c6-b442-258e027e05b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:01.012501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c00aff-99c9-41c6-b442-258e027e05b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:10:01.012501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c00aff-99c9-41c6-b442-258e027e05b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:01.014146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150220ec-7091-4868-8b1d-6f11d2e1c70c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:01.016348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c00aff-99c9-41c6-b442-258e027e05b4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:01.021744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8319a34-3c84-43ec-ad0d-34f1d8f77498 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:01.022901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8319a34-3c84-43ec-ad0d-34f1d8f77498 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:01.023566 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8319a34-3c84-43ec-ad0d-34f1d8f77498 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:01.024136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8319a34-3c84-43ec-ad0d-34f1d8f77498 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:01.037827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 846/1696] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:10:01.037827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62da2cc-d93e-450e-ba0d-cfc36d57f445 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:01.037827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62da2cc-d93e-450e-ba0d-cfc36d57f445 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:01.037827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62da2cc-d93e-450e-ba0d-cfc36d57f445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:01.047700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8319a34-3c84-43ec-ad0d-34f1d8f77498 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:01.052041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8319a34-3c84-43ec-ad0d-34f1d8f77498 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:01.084639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 851/1697] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:01.085964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8319a34-3c84-43ec-ad0d-34f1d8f77498 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:01.139410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d62da2cc-d93e-450e-ba0d-cfc36d57f445 None None] MFA Rules not processed for user `c48d41dcf9b344a8b2fec77f4dd75fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:01.144177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8319a34-3c84-43ec-ad0d-34f1d8f77498 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:01.145384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d62da2cc-d93e-450e-ba0d-cfc36d57f445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:01.172954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878959e3-eba4-41fb-b368-31cde1780e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:01.178308 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878959e3-eba4-41fb-b368-31cde1780e0d admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:01.182203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878959e3-eba4-41fb-b368-31cde1780e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:01.182813 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878959e3-eba4-41fb-b368-31cde1780e0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:01.186004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878959e3-eba4-41fb-b368-31cde1780e0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:01.197103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878959e3-eba4-41fb-b368-31cde1780e0d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:01.357106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 852/1698] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:01 2022] POST /identity/v3/projects => generated 387 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:01.363516 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878959e3-eba4-41fb-b368-31cde1780e0d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:01.397800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 847/1699] 10.222.0.13 () {58 vars in 1059 bytes} [Thu Jul 21 09:10:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:01.433401 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62da2cc-d93e-450e-ba0d-cfc36d57f445 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:01.455176 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62da2cc-d93e-450e-ba0d-cfc36d57f445 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:01.566384 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878959e3-eba4-41fb-b368-31cde1780e0d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:01.586505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7065c-7166-4494-a02c-aa817b4b7c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:01.587686 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7065c-7166-4494-a02c-aa817b4b7c7d admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:01.588241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7065c-7166-4494-a02c-aa817b4b7c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:01.609153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7065c-7166-4494-a02c-aa817b4b7c7d admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:01.610567 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7065c-7166-4494-a02c-aa817b4b7c7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:01.615514 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7065c-7166-4494-a02c-aa817b4b7c7d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:01.686487 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 853/1700] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:01 2022] POST /identity/v3/users => generated 294 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:01.690823 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7065c-7166-4494-a02c-aa817b4b7c7d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:01.694803 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3eb7065c-7166-4494-a02c-aa817b4b7c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:01.736203 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313aa2f8-7409-4907-a751-c266029f9fac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:01.740110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aa2f8-7409-4907-a751-c266029f9fac service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:01.740514 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aa2f8-7409-4907-a751-c266029f9fac service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:01.740819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aa2f8-7409-4907-a751-c266029f9fac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:01.779721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:01.783423 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:01.847185 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7065c-7166-4494-a02c-aa817b4b7c7d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:01.854517 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe85f2-a917-4520-917a-891a14d1c396 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:01.856827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe85f2-a917-4520-917a-891a14d1c396 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:01.857315 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe85f2-a917-4520-917a-891a14d1c396 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:01.857817 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe85f2-a917-4520-917a-891a14d1c396 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:01.861561 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1abe85f2-a917-4520-917a-891a14d1c396 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:01.947689 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313aa2f8-7409-4907-a751-c266029f9fac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:01.953262 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313aa2f8-7409-4907-a751-c266029f9fac service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:01.998179 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 848/1701] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:10:02.001757 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313aa2f8-7409-4907-a751-c266029f9fac service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:02.009606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe85f2-a917-4520-917a-891a14d1c396 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:02.014610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe85f2-a917-4520-917a-891a14d1c396 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:02.041644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313aa2f8-7409-4907-a751-c266029f9fac service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:02.062702 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 854/1702] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:02.064779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe85f2-a917-4520-917a-891a14d1c396 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:02.067494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d16457-c643-4ef1-aa9e-dff9414af118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:02.070853 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1abe85f2-a917-4520-917a-891a14d1c396 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:02.071307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d16457-c643-4ef1-aa9e-dff9414af118 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:02.071986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d16457-c643-4ef1-aa9e-dff9414af118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:02.072587 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d16457-c643-4ef1-aa9e-dff9414af118 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:02.077327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d16457-c643-4ef1-aa9e-dff9414af118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:02.082316 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d16457-c643-4ef1-aa9e-dff9414af118 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:02.150352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe85f2-a917-4520-917a-891a14d1c396 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:02.160347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 849/1703] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:02.161542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d16457-c643-4ef1-aa9e-dff9414af118 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:02.164811 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440a8073-29f9-4d81-82f2-db88e969cb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:02.166017 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a8073-29f9-4d81-82f2-db88e969cb8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:02.166377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a8073-29f9-4d81-82f2-db88e969cb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:02.166728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a8073-29f9-4d81-82f2-db88e969cb8d admin admin] PATH_INFO: `/v3/users/d39267e4092b487bb2e51656e34e46dc` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:02.169308 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a8073-29f9-4d81-82f2-db88e969cb8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39267e4092b487bb2e51656e34e46dc)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:02.174262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440a8073-29f9-4d81-82f2-db88e969cb8d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:02.192369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d16457-c643-4ef1-aa9e-dff9414af118 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:02.212113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d2109-135c-44b9-8a0d-36bc462d1efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:02.212894 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d2109-135c-44b9-8a0d-36bc462d1efb admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:02.218952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d2109-135c-44b9-8a0d-36bc462d1efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:02.219394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d2109-135c-44b9-8a0d-36bc462d1efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:02.220312 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d2109-135c-44b9-8a0d-36bc462d1efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:02.222036 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d2109-135c-44b9-8a0d-36bc462d1efb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:02.253188 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 850/1704] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:02.257678 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d2109-135c-44b9-8a0d-36bc462d1efb admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:02.285896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d2109-135c-44b9-8a0d-36bc462d1efb admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:02.303888 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fa382-934c-49f0-8021-e729dda41697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:02.306205 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fa382-934c-49f0-8021-e729dda41697 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:02.307614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fa382-934c-49f0-8021-e729dda41697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:02.308672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fa382-934c-49f0-8021-e729dda41697 admin admin] PATH_INFO: `/v3/projects/1da5c266da9140ea8807eec96cd02a42/users/d795b26ab10b45d39f7ca9b7e8466c07/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:02.348361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-440a8073-29f9-4d81-82f2-db88e969cb8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39267e4092b487bb2e51656e34e46dc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:02.349950 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-440a8073-29f9-4d81-82f2-db88e969cb8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39267e4092b487bb2e51656e34e46dc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:02.378469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fa382-934c-49f0-8021-e729dda41697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da5c266da9140ea8807eec96cd02a42, user_id=d795b26ab10b45d39f7ca9b7e8466c07, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:02.384089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-440a8073-29f9-4d81-82f2-db88e969cb8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39267e4092b487bb2e51656e34e46dc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:02.388094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fa382-934c-49f0-8021-e729dda41697 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:02.434987 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-440a8073-29f9-4d81-82f2-db88e969cb8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39267e4092b487bb2e51656e34e46dc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:02.470678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-440a8073-29f9-4d81-82f2-db88e969cb8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39267e4092b487bb2e51656e34e46dc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:02.478035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 851/1705] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:02 2022] PUT /identity/v3/projects/1da5c266da9140ea8807eec96cd02a42/users/d795b26ab10b45d39f7ca9b7e8466c07/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:02.493866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034faf0-3b2f-46fa-991f-50c2b743b4cb None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:02.494425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034faf0-3b2f-46fa-991f-50c2b743b4cb None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:02.494920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034faf0-3b2f-46fa-991f-50c2b743b4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:02.498266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 855/1706] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:10:02 2022] DELETE /identity/v3/users/d39267e4092b487bb2e51656e34e46dc => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:02.504204 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29555e-5361-48d2-ac31-4f5e8935de5d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:02.504623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29555e-5361-48d2-ac31-4f5e8935de5d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:02.505136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29555e-5361-48d2-ac31-4f5e8935de5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:02.529874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3a29555e-5361-48d2-ac31-4f5e8935de5d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:02.531420 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3a29555e-5361-48d2-ac31-4f5e8935de5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:02.598603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c034faf0-3b2f-46fa-991f-50c2b743b4cb None None] MFA Rules not processed for user `d795b26ab10b45d39f7ca9b7e8466c07`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:02.600664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c034faf0-3b2f-46fa-991f-50c2b743b4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:02.626633 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:02.665498 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 856/1707] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:02.691455 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29555e-5361-48d2-ac31-4f5e8935de5d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:02.695064 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3a29555e-5361-48d2-ac31-4f5e8935de5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:02.825054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 852/1708] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:10:02.826408 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034faf0-3b2f-46fa-991f-50c2b743b4cb None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:02.835236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c034faf0-3b2f-46fa-991f-50c2b743b4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:02.848462 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29555e-5361-48d2-ac31-4f5e8935de5d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:02.860818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7903ef3-8271-4974-aa5c-217c8f97f300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:02.861539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7903ef3-8271-4974-aa5c-217c8f97f300 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:02.861824 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7903ef3-8271-4974-aa5c-217c8f97f300 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:02.862105 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7903ef3-8271-4974-aa5c-217c8f97f300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:02.873190 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034faf0-3b2f-46fa-991f-50c2b743b4cb None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:02.886622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7903ef3-8271-4974-aa5c-217c8f97f300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:02.892917 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9172e0-3e58-4115-b0d8-3e19d85b316f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:02.894022 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7903ef3-8271-4974-aa5c-217c8f97f300 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:02.894781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9172e0-3e58-4115-b0d8-3e19d85b316f admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:02.897429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9172e0-3e58-4115-b0d8-3e19d85b316f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:02.898184 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9172e0-3e58-4115-b0d8-3e19d85b316f admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:02.909483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9172e0-3e58-4115-b0d8-3e19d85b316f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:02.914750 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9172e0-3e58-4115-b0d8-3e19d85b316f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:02.921597 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 857/1709] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:10:03.010352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7903ef3-8271-4974-aa5c-217c8f97f300 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:03.010882 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 853/1710] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:02 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:03.026224 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7903ef3-8271-4974-aa5c-217c8f97f300 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:03.197584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed16ce9-219b-4698-abf9-7de5f54505b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:03.198377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed16ce9-219b-4698-abf9-7de5f54505b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:03.198976 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed16ce9-219b-4698-abf9-7de5f54505b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:03.199068 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed16ce9-219b-4698-abf9-7de5f54505b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:03.199867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed16ce9-219b-4698-abf9-7de5f54505b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:03.205990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed16ce9-219b-4698-abf9-7de5f54505b0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:03.253227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 858/1711] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:03 2022] POST /identity/v3/users => generated 299 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:10:03.255698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9172e0-3e58-4115-b0d8-3e19d85b316f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:03.278482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9172e0-3e58-4115-b0d8-3e19d85b316f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:03.298560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0f3de-288e-4f78-812e-b51b5aba1fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:03.299765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0f3de-288e-4f78-812e-b51b5aba1fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:03.300608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0f3de-288e-4f78-812e-b51b5aba1fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:03.300771 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0f3de-288e-4f78-812e-b51b5aba1fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:03.302455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0f3de-288e-4f78-812e-b51b5aba1fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:03.307566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0f3de-288e-4f78-812e-b51b5aba1fa2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:03.358982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed16ce9-219b-4698-abf9-7de5f54505b0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:03.365491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 854/1712] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:03.373970 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed16ce9-219b-4698-abf9-7de5f54505b0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:03.407363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a251a5-da01-47ac-bbb2-8c4431c3a14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:03.408617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a251a5-da01-47ac-bbb2-8c4431c3a14b admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:03.408903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a251a5-da01-47ac-bbb2-8c4431c3a14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:03.409177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a251a5-da01-47ac-bbb2-8c4431c3a14b admin admin] PATH_INFO: `/v3/projects/a1fd17052dcf4398a5137ad1f89703a7/users/513d774063194482a775c47e24ad7b5b/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:03.463683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a251a5-da01-47ac-bbb2-8c4431c3a14b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1fd17052dcf4398a5137ad1f89703a7, user_id=513d774063194482a775c47e24ad7b5b, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:03.475152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a251a5-da01-47ac-bbb2-8c4431c3a14b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:03.507648 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 859/1713] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:03 2022] PUT /identity/v3/projects/a1fd17052dcf4398a5137ad1f89703a7/users/513d774063194482a775c47e24ad7b5b/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:10:03.515959 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0f3de-288e-4f78-812e-b51b5aba1fa2 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:03.548924 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0f3de-288e-4f78-812e-b51b5aba1fa2 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:03.624160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:03.789742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67e224-a4da-4334-87f6-bbf0951a2cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:03.790453 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67e224-a4da-4334-87f6-bbf0951a2cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:03.790727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67e224-a4da-4334-87f6-bbf0951a2cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:03.791004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67e224-a4da-4334-87f6-bbf0951a2cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:03.791761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67e224-a4da-4334-87f6-bbf0951a2cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:03.794760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67e224-a4da-4334-87f6-bbf0951a2cb0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:03.825012 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a251a5-da01-47ac-bbb2-8c4431c3a14b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:03.828384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 855/1714] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:03.866329 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a251a5-da01-47ac-bbb2-8c4431c3a14b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:03.890896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958c20d-9797-4af2-88f4-6195dc7c8579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:03.893415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958c20d-9797-4af2-88f4-6195dc7c8579 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:03.901315 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958c20d-9797-4af2-88f4-6195dc7c8579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:03.903819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958c20d-9797-4af2-88f4-6195dc7c8579 admin admin] PATH_INFO: `/v3/projects/a1fd17052dcf4398a5137ad1f89703a7/users/513d774063194482a775c47e24ad7b5b/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:03.911173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958c20d-9797-4af2-88f4-6195dc7c8579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1fd17052dcf4398a5137ad1f89703a7, user_id=513d774063194482a775c47e24ad7b5b, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:03.914883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958c20d-9797-4af2-88f4-6195dc7c8579 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:04.007464 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0b5c5-d57d-4e9b-9526-a26ce2bac9f1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:04.007818 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0b5c5-d57d-4e9b-9526-a26ce2bac9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:04.008463 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0b5c5-d57d-4e9b-9526-a26ce2bac9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:04.010601 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 860/1715] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:03 2022] PUT /identity/v3/projects/a1fd17052dcf4398a5137ad1f89703a7/users/513d774063194482a775c47e24ad7b5b/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:04.111251 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5ef0b5c5-d57d-4e9b-9526-a26ce2bac9f1 None None] MFA Rules not processed for user `513d774063194482a775c47e24ad7b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:04.112624 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5ef0b5c5-d57d-4e9b-9526-a26ce2bac9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:04.338898 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 856/1716] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:04.357273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958c20d-9797-4af2-88f4-6195dc7c8579 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:04.359102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c958c20d-9797-4af2-88f4-6195dc7c8579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:04.502956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:04.560877 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958c20d-9797-4af2-88f4-6195dc7c8579 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:04.578203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c18217-5869-45d8-8417-ca690aeb2ad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:04.578812 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c18217-5869-45d8-8417-ca690aeb2ad2 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:04.579097 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c18217-5869-45d8-8417-ca690aeb2ad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:04.579376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c18217-5869-45d8-8417-ca690aeb2ad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:04.608952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c18217-5869-45d8-8417-ca690aeb2ad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:04.610480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c18217-5869-45d8-8417-ca690aeb2ad2 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:04.639342 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 861/1717] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:04.710489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0b5c5-d57d-4e9b-9526-a26ce2bac9f1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:04.847030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0b5c5-d57d-4e9b-9526-a26ce2bac9f1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:04.853777 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5dc90-27e3-4bb7-998b-b5db817502b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:04.855579 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5dc90-27e3-4bb7-998b-b5db817502b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:04.856569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5dc90-27e3-4bb7-998b-b5db817502b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:04.856858 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5dc90-27e3-4bb7-998b-b5db817502b0 admin admin] PATH_INFO: `/v3/users/d795b26ab10b45d39f7ca9b7e8466c07` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:04.871257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5dc90-27e3-4bb7-998b-b5db817502b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d795b26ab10b45d39f7ca9b7e8466c07)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:04.874999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5dc90-27e3-4bb7-998b-b5db817502b0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:05.077499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:05.081953 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c4b5dc90-27e3-4bb7-998b-b5db817502b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd795b26ab10b45d39f7ca9b7e8466c07'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:05.108266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c4b5dc90-27e3-4bb7-998b-b5db817502b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd795b26ab10b45d39f7ca9b7e8466c07'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:05.119315 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c4b5dc90-27e3-4bb7-998b-b5db817502b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd795b26ab10b45d39f7ca9b7e8466c07'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:05.147426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c4b5dc90-27e3-4bb7-998b-b5db817502b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd795b26ab10b45d39f7ca9b7e8466c07'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:05.152996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c4b5dc90-27e3-4bb7-998b-b5db817502b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd795b26ab10b45d39f7ca9b7e8466c07'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:05.179757 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 857/1718] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:10:04 2022] DELETE /identity/v3/users/d795b26ab10b45d39f7ca9b7e8466c07 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:05.217434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d5c83-5e35-4ae8-8ddf-16b85bc79666 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:05.218010 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d5c83-5e35-4ae8-8ddf-16b85bc79666 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:05.218505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d5c83-5e35-4ae8-8ddf-16b85bc79666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:05.325766 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-741d5c83-5e35-4ae8-8ddf-16b85bc79666 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:05.328061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-741d5c83-5e35-4ae8-8ddf-16b85bc79666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:05.570426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 862/1719] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:05.649024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5dc90-27e3-4bb7-998b-b5db817502b0 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:05.651097 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c4b5dc90-27e3-4bb7-998b-b5db817502b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:05.794856 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5dc90-27e3-4bb7-998b-b5db817502b0 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:05.800276 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b243129-9e6d-47d6-adb7-df381633fed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:05.802458 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b243129-9e6d-47d6-adb7-df381633fed2 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:05.805017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b243129-9e6d-47d6-adb7-df381633fed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:05.805603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b243129-9e6d-47d6-adb7-df381633fed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:05.826983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b243129-9e6d-47d6-adb7-df381633fed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:05.830847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b243129-9e6d-47d6-adb7-df381633fed2 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:05.854402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 858/1720] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:07.298475 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d5c83-5e35-4ae8-8ddf-16b85bc79666 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:07.301311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-741d5c83-5e35-4ae8-8ddf-16b85bc79666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:07.348388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d5c83-5e35-4ae8-8ddf-16b85bc79666 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:07.358837 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b243129-9e6d-47d6-adb7-df381633fed2 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:07.362309 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5b243129-9e6d-47d6-adb7-df381633fed2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:07.374112 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:07.378863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:07.383071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:07.384813 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e admin admin] PATH_INFO: `/v3/projects/1da5c266da9140ea8807eec96cd02a42` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:07.388207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da5c266da9140ea8807eec96cd02a42)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:07.395948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:07.445244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b243129-9e6d-47d6-adb7-df381633fed2 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:07.478534 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:07.481556 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:07.482052 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:07.489205 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3 admin admin] PATH_INFO: `/v3/projects/25cd385fd699436b954f9b5c38353386` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:07.496763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25cd385fd699436b954f9b5c38353386)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:07.501238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:07.592452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da5c266da9140ea8807eec96cd02a42', 'request_id': 'req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:07.624966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da5c266da9140ea8807eec96cd02a42', 'request_id': 'req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:07.717518 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da5c266da9140ea8807eec96cd02a42', 'request_id': 'req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:07.785343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e admin admin] The token cache is being invalidate because project 1da5c266da9140ea8807eec96cd02a42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:07.785855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:07.790017 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:07.807674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 863/1721] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:10:07 2022] DELETE /identity/v3/projects/1da5c266da9140ea8807eec96cd02a42 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:10:07.810093 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:07.816403 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25cd385fd699436b954f9b5c38353386', 'request_id': 'req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:07.819661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:07.834944 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25cd385fd699436b954f9b5c38353386', 'request_id': 'req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:07.853701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25cd385fd699436b954f9b5c38353386', 'request_id': 'req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:07.890698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3 admin admin] The token cache is being invalidate because project 25cd385fd699436b954f9b5c38353386 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:07.890988 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:07.894216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:07.899458 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 859/1722] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:10:07 2022] DELETE /identity/v3/projects/25cd385fd699436b954f9b5c38353386 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:07.905147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:07.907028 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:08.075237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:08.143345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed11d3f-afdc-46b0-b3d2-44aa1ce3edd3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:08.165023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cce34-602e-428f-9305-1d2ab005ca03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:08.166038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cce34-602e-428f-9305-1d2ab005ca03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:08.166378 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cce34-602e-428f-9305-1d2ab005ca03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:08.166660 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cce34-602e-428f-9305-1d2ab005ca03 admin admin] PATH_INFO: `/v3/users/21a7a758f56448e8a4133818b50fa166` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:08.170726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cce34-602e-428f-9305-1d2ab005ca03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21a7a758f56448e8a4133818b50fa166)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:08.177461 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cce34-602e-428f-9305-1d2ab005ca03 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:08.184856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d73f3-80e3-4cd4-83c0-4ab8d707b45e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:08.203034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b330777-8525-4839-a446-6a06e74ce7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:08.204443 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b330777-8525-4839-a446-6a06e74ce7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:08.204955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b330777-8525-4839-a446-6a06e74ce7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:08.205466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b330777-8525-4839-a446-6a06e74ce7e5 admin admin] PATH_INFO: `/v3/users/513d774063194482a775c47e24ad7b5b` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:08.217732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b330777-8525-4839-a446-6a06e74ce7e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=513d774063194482a775c47e24ad7b5b)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:08.221785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b330777-8525-4839-a446-6a06e74ce7e5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:08.298733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5c3cce34-602e-428f-9305-1d2ab005ca03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21a7a758f56448e8a4133818b50fa166'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:08.299038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:08.329815 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5c3cce34-602e-428f-9305-1d2ab005ca03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21a7a758f56448e8a4133818b50fa166'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:08.347852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5c3cce34-602e-428f-9305-1d2ab005ca03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21a7a758f56448e8a4133818b50fa166'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:08.409235 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5c3cce34-602e-428f-9305-1d2ab005ca03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21a7a758f56448e8a4133818b50fa166'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:08.413412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5c3cce34-602e-428f-9305-1d2ab005ca03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21a7a758f56448e8a4133818b50fa166'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:08.452143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5b330777-8525-4839-a446-6a06e74ce7e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '513d774063194482a775c47e24ad7b5b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:08.454887 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5b330777-8525-4839-a446-6a06e74ce7e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '513d774063194482a775c47e24ad7b5b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:08.455570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 860/1723] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:10:07 2022] DELETE /identity/v3/users/21a7a758f56448e8a4133818b50fa166 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:08.457480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cce34-602e-428f-9305-1d2ab005ca03 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:08.465409 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5c3cce34-602e-428f-9305-1d2ab005ca03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:08.500564 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5b330777-8525-4839-a446-6a06e74ce7e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '513d774063194482a775c47e24ad7b5b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:08.586188 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5b330777-8525-4839-a446-6a06e74ce7e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '513d774063194482a775c47e24ad7b5b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:08.637549 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5b330777-8525-4839-a446-6a06e74ce7e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '513d774063194482a775c47e24ad7b5b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:08.650325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 864/1724] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:10:07 2022] DELETE /identity/v3/users/513d774063194482a775c47e24ad7b5b => generated 0 bytes in 841 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:10:08.826174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cce34-602e-428f-9305-1d2ab005ca03 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:08.962296 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb8c87-970b-4bf7-adff-2c1b376867b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:08.964600 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb8c87-970b-4bf7-adff-2c1b376867b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:08.965238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb8c87-970b-4bf7-adff-2c1b376867b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:08.966336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb8c87-970b-4bf7-adff-2c1b376867b9 admin admin] PATH_INFO: `/v3/users/d3857f0ca1f74afe9c42532d2fcabf9b` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:08.968149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb8c87-970b-4bf7-adff-2c1b376867b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3857f0ca1f74afe9c42532d2fcabf9b)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:08.970463 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb8c87-970b-4bf7-adff-2c1b376867b9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:08.993955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:09.134668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-59fb8c87-970b-4bf7-adff-2c1b376867b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3857f0ca1f74afe9c42532d2fcabf9b'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:09.174654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-59fb8c87-970b-4bf7-adff-2c1b376867b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3857f0ca1f74afe9c42532d2fcabf9b'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:09.232481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-59fb8c87-970b-4bf7-adff-2c1b376867b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3857f0ca1f74afe9c42532d2fcabf9b'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:09.290695 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-59fb8c87-970b-4bf7-adff-2c1b376867b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3857f0ca1f74afe9c42532d2fcabf9b'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:09.291894 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-59fb8c87-970b-4bf7-adff-2c1b376867b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3857f0ca1f74afe9c42532d2fcabf9b'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:09.309878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 861/1725] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:10:08 2022] DELETE /identity/v3/users/d3857f0ca1f74afe9c42532d2fcabf9b => generated 0 bytes in 853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:09.318050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692b53d-b07e-4de0-ad3d-f4c2582ee7fd None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:09.318599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692b53d-b07e-4de0-ad3d-f4c2582ee7fd None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:09.319063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692b53d-b07e-4de0-ad3d-f4c2582ee7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:09.430449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-2692b53d-b07e-4de0-ad3d-f4c2582ee7fd None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:09.437314 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2692b53d-b07e-4de0-ad3d-f4c2582ee7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:09.607305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 865/1726] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:09.646438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb8c87-970b-4bf7-adff-2c1b376867b9 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:09.648585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-59fb8c87-970b-4bf7-adff-2c1b376867b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:09.858273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb8c87-970b-4bf7-adff-2c1b376867b9 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:09.866266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53e944-e7a2-4a43-a5b6-dffa13e1824a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:09.867154 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53e944-e7a2-4a43-a5b6-dffa13e1824a service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:09.867450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53e944-e7a2-4a43-a5b6-dffa13e1824a service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:09.867733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53e944-e7a2-4a43-a5b6-dffa13e1824a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:09.905847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53e944-e7a2-4a43-a5b6-dffa13e1824a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:09.913268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53e944-e7a2-4a43-a5b6-dffa13e1824a service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:09.967178 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 862/1727] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:10:10.520903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eeac3-1890-4608-b459-49d40996133b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:10.521467 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eeac3-1890-4608-b459-49d40996133b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:10.522414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eeac3-1890-4608-b459-49d40996133b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:10.613050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-640eeac3-1890-4608-b459-49d40996133b None None] MFA Rules not processed for user `be1372c20ad94145b775663a9037d55a`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:10.616276 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-640eeac3-1890-4608-b459-49d40996133b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:10.769413 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 866/1728] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:10.785997 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1074e-6ac8-4d09-a57d-84cee2d7e206 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:10.787054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1074e-6ac8-4d09-a57d-84cee2d7e206 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:10.787054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1074e-6ac8-4d09-a57d-84cee2d7e206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:10.849728 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6ef1074e-6ac8-4d09-a57d-84cee2d7e206 None None] MFA Rules not processed for user `be1372c20ad94145b775663a9037d55a`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:10.852291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6ef1074e-6ac8-4d09-a57d-84cee2d7e206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:10.883045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 863/1729] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:10.925529 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d86e84-385b-4576-88dc-4b6795a06e3e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:10.926275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d86e84-385b-4576-88dc-4b6795a06e3e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:10.926780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d86e84-385b-4576-88dc-4b6795a06e3e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:10.929365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 867/1730] 10.222.0.13 () {54 vars in 962 bytes} [Thu Jul 21 09:10:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:10:10.954986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773af7d-1ed0-4181-82df-75fde1bb928b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:10.955486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773af7d-1ed0-4181-82df-75fde1bb928b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:10.955996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773af7d-1ed0-4181-82df-75fde1bb928b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:11.015097 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6773af7d-1ed0-4181-82df-75fde1bb928b None None] MFA Rules not processed for user `c48d41dcf9b344a8b2fec77f4dd75fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:11.022892 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6773af7d-1ed0-4181-82df-75fde1bb928b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:11.289212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 864/1731] 10.222.0.13 () {58 vars in 1059 bytes} [Thu Jul 21 09:10:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:11.297650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d86e84-385b-4576-88dc-4b6795a06e3e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:11.357556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d86e84-385b-4576-88dc-4b6795a06e3e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:11.365775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc255f-1548-4c35-8e38-ff3b38013231 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:11.366567 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc255f-1548-4c35-8e38-ff3b38013231 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:11.366856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc255f-1548-4c35-8e38-ff3b38013231 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:11.367135 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc255f-1548-4c35-8e38-ff3b38013231 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:11.372800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773af7d-1ed0-4181-82df-75fde1bb928b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:11.375191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6773af7d-1ed0-4181-82df-75fde1bb928b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:11.379575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc255f-1548-4c35-8e38-ff3b38013231 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:11.381557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc255f-1548-4c35-8e38-ff3b38013231 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:11.393945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 868/1732] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:11.414833 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773af7d-1ed0-4181-82df-75fde1bb928b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:11.440286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbd883-969c-4b21-895a-5eae9c88e483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:11.440286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbd883-969c-4b21-895a-5eae9c88e483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:11.440286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbd883-969c-4b21-895a-5eae9c88e483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:11.440286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbd883-969c-4b21-895a-5eae9c88e483 admin admin] PATH_INFO: `/v3/projects/a1fd17052dcf4398a5137ad1f89703a7` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:11.440576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbd883-969c-4b21-895a-5eae9c88e483 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1fd17052dcf4398a5137ad1f89703a7)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:11.442839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82516f1-2af4-4cb0-9eba-b24b7e7f9616 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:11.443402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82516f1-2af4-4cb0-9eba-b24b7e7f9616 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:11.443946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82516f1-2af4-4cb0-9eba-b24b7e7f9616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:11.445621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbd883-969c-4b21-895a-5eae9c88e483 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:11.466132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:11.490950 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e82516f1-2af4-4cb0-9eba-b24b7e7f9616 None None] MFA Rules not processed for user `675378ce4e594727a59049f6ce476ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:11.492526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e82516f1-2af4-4cb0-9eba-b24b7e7f9616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:11.498659 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:11.624055 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-86dbd883-969c-4b21-895a-5eae9c88e483 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1fd17052dcf4398a5137ad1f89703a7', 'request_id': 'req-86dbd883-969c-4b21-895a-5eae9c88e483'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:11.631539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 869/1733] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:11.635021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adb85c-4f32-46e1-8f20-5a4bd6e580e2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:11.635674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adb85c-4f32-46e1-8f20-5a4bd6e580e2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:11.636391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adb85c-4f32-46e1-8f20-5a4bd6e580e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:11.658922 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-86dbd883-969c-4b21-895a-5eae9c88e483 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1fd17052dcf4398a5137ad1f89703a7', 'request_id': 'req-86dbd883-969c-4b21-895a-5eae9c88e483'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:11.662476 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-50adb85c-4f32-46e1-8f20-5a4bd6e580e2 None None] MFA Rules not processed for user `675378ce4e594727a59049f6ce476ce0`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:11.666266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-50adb85c-4f32-46e1-8f20-5a4bd6e580e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:11.677116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-86dbd883-969c-4b21-895a-5eae9c88e483 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1fd17052dcf4398a5137ad1f89703a7', 'request_id': 'req-86dbd883-969c-4b21-895a-5eae9c88e483'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:11.724916 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-86dbd883-969c-4b21-895a-5eae9c88e483 admin admin] The token cache is being invalidate because project a1fd17052dcf4398a5137ad1f89703a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:11.725355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-86dbd883-969c-4b21-895a-5eae9c88e483 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:11.732619 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-86dbd883-969c-4b21-895a-5eae9c88e483 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:11.736001 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 865/1734] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:10:11 2022] DELETE /identity/v3/projects/a1fd17052dcf4398a5137ad1f89703a7 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:11.742295 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e72fa-889f-4d87-aac5-ce867c660c87 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:11.742804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e72fa-889f-4d87-aac5-ce867c660c87 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:11.743313 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e72fa-889f-4d87-aac5-ce867c660c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:11.751872 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 870/1735] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:11.770621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50adb85c-4f32-46e1-8f20-5a4bd6e580e2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:11.776002 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-50adb85c-4f32-46e1-8f20-5a4bd6e580e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:11.818522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-994e72fa-889f-4d87-aac5-ce867c660c87 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:11.829042 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-994e72fa-889f-4d87-aac5-ce867c660c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:12.013429 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50adb85c-4f32-46e1-8f20-5a4bd6e580e2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:12.031277 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c965385-9aa8-4ae5-9147-457083f67c03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:12.032405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c965385-9aa8-4ae5-9147-457083f67c03 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:12.032814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c965385-9aa8-4ae5-9147-457083f67c03 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:12.033207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c965385-9aa8-4ae5-9147-457083f67c03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:12.037515 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8c965385-9aa8-4ae5-9147-457083f67c03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:12.057306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 866/1736] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:12.060020 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6737d5d-043c-4c5b-bdb7-67db4cbf8635 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:12.060475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6737d5d-043c-4c5b-bdb7-67db4cbf8635 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:12.060880 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6737d5d-043c-4c5b-bdb7-67db4cbf8635 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:12.062792 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 867/1737] 10.222.0.13 () {54 vars in 912 bytes} [Thu Jul 21 09:10:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:10:12.079596 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c965385-9aa8-4ae5-9147-457083f67c03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:12.081646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c965385-9aa8-4ae5-9147-457083f67c03 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:12.090180 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356616e4-2f91-4163-b0f6-9e467c977796 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:12.100674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356616e4-2f91-4163-b0f6-9e467c977796 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:12.103139 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356616e4-2f91-4163-b0f6-9e467c977796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:12.116049 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 871/1738] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:10:12.117781 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f764b-ffc9-47f0-8138-fa916f7722d5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:12.118241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f764b-ffc9-47f0-8138-fa916f7722d5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:12.118861 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f764b-ffc9-47f0-8138-fa916f7722d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:12.172524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-119f764b-ffc9-47f0-8138-fa916f7722d5 None None] MFA Rules not processed for user `c48d41dcf9b344a8b2fec77f4dd75fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:12.173814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-119f764b-ffc9-47f0-8138-fa916f7722d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:12.194036 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 872/1739] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jul 21 09:10:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:12.224379 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f764b-ffc9-47f0-8138-fa916f7722d5 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:12.225671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-356616e4-2f91-4163-b0f6-9e467c977796 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:12.226189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-119f764b-ffc9-47f0-8138-fa916f7722d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:12.232569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-356616e4-2f91-4163-b0f6-9e467c977796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:12.277301 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 868/1740] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:12.284018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:12.285809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356616e4-2f91-4163-b0f6-9e467c977796 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:12.317123 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356616e4-2f91-4163-b0f6-9e467c977796 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:12.329793 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062144c-7b8f-440e-a8ea-19d5508d140f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:12.330773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062144c-7b8f-440e-a8ea-19d5508d140f admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:12.331357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062144c-7b8f-440e-a8ea-19d5508d140f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:12.331750 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062144c-7b8f-440e-a8ea-19d5508d140f admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:12.334674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062144c-7b8f-440e-a8ea-19d5508d140f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:10:12.335402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062144c-7b8f-440e-a8ea-19d5508d140f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:12.339581 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062144c-7b8f-440e-a8ea-19d5508d140f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:12.377780 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 869/1741] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:10:12.379636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5696f3-e883-445f-8c93-39a2f7b32dbd None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:12.382036 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5696f3-e883-445f-8c93-39a2f7b32dbd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:12.383302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5696f3-e883-445f-8c93-39a2f7b32dbd None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:12.390767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 870/1742] 10.222.0.13 () {54 vars in 912 bytes} [Thu Jul 21 09:10:12 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:10:12.391826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5696f3-e883-445f-8c93-39a2f7b32dbd None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:12.428350 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5696f3-e883-445f-8c93-39a2f7b32dbd None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:12.438119 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a16fb-b9a0-4890-b162-8aae245fcdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:12.438908 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a16fb-b9a0-4890-b162-8aae245fcdb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:12.439259 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a16fb-b9a0-4890-b162-8aae245fcdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:12.439606 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a16fb-b9a0-4890-b162-8aae245fcdb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:12.441707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a16fb-b9a0-4890-b162-8aae245fcdb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:12.446365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f764b-ffc9-47f0-8138-fa916f7722d5 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:12.460118 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a16fb-b9a0-4890-b162-8aae245fcdb1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:12.474150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512ee07-e25a-4208-94e8-ae9a26874119 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e053b6b25a4e4a20a05e6ffa6683222b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:12.477352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512ee07-e25a-4208-94e8-ae9a26874119 service glance] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:12.478714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512ee07-e25a-4208-94e8-ae9a26874119 service glance] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:12.479123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512ee07-e25a-4208-94e8-ae9a26874119 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:12.514402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512ee07-e25a-4208-94e8-ae9a26874119 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:12.516128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512ee07-e25a-4208-94e8-ae9a26874119 service glance] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:12.541663 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 871/1743] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:12 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:12.551698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cf500-60dc-47e2-bb43-b4fb5dc4a1b7 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:12.552732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cf500-60dc-47e2-bb43-b4fb5dc4a1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:12.553507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cf500-60dc-47e2-bb43-b4fb5dc4a1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:12.567074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 873/1744] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:12.568431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512ee07-e25a-4208-94e8-ae9a26874119 service glance] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:12.619653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512ee07-e25a-4208-94e8-ae9a26874119 service glance] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:12.628974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a58cf500-60dc-47e2-bb43-b4fb5dc4a1b7 None None] MFA Rules not processed for user `c48d41dcf9b344a8b2fec77f4dd75fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:12.631671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a58cf500-60dc-47e2-bb43-b4fb5dc4a1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:12.879040 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8b230-608b-45dc-8385-a019d6b4986e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:12.880449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8b230-608b-45dc-8385-a019d6b4986e admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:12.880938 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8b230-608b-45dc-8385-a019d6b4986e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:12.881938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:12.887726 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8b230-608b-45dc-8385-a019d6b4986e admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:12.891885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8b230-608b-45dc-8385-a019d6b4986e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:12.905024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8b230-608b-45dc-8385-a019d6b4986e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:12.934259 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 872/1745] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jul 21 09:10:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:12.936532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cf500-60dc-47e2-bb43-b4fb5dc4a1b7 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:12.939036 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a58cf500-60dc-47e2-bb43-b4fb5dc4a1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:12.962548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 874/1746] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:12 2022] POST /identity/v3/users => generated 290 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:12.967314 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8b230-608b-45dc-8385-a019d6b4986e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:12.981590 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cf500-60dc-47e2-bb43-b4fb5dc4a1b7 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:13.000601 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd73a68-c927-467e-a6bf-1723f3c9a29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:13.001489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd73a68-c927-467e-a6bf-1723f3c9a29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:13.001937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd73a68-c927-467e-a6bf-1723f3c9a29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:13.002365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd73a68-c927-467e-a6bf-1723f3c9a29f admin admin] PATH_INFO: `/v3/projects/7b6bf14245944a619c58b2619c87471a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:13.004327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd73a68-c927-467e-a6bf-1723f3c9a29f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b6bf14245944a619c58b2619c87471a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:13.006491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd73a68-c927-467e-a6bf-1723f3c9a29f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:13.133618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9bd73a68-c927-467e-a6bf-1723f3c9a29f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b6bf14245944a619c58b2619c87471a', 'request_id': 'req-9bd73a68-c927-467e-a6bf-1723f3c9a29f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:13.138968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8b230-608b-45dc-8385-a019d6b4986e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:13.162717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9bd73a68-c927-467e-a6bf-1723f3c9a29f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b6bf14245944a619c58b2619c87471a', 'request_id': 'req-9bd73a68-c927-467e-a6bf-1723f3c9a29f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:13.182077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9bd73a68-c927-467e-a6bf-1723f3c9a29f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b6bf14245944a619c58b2619c87471a', 'request_id': 'req-9bd73a68-c927-467e-a6bf-1723f3c9a29f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:13.207868 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9bd73a68-c927-467e-a6bf-1723f3c9a29f admin admin] The token cache is being invalidate because project 7b6bf14245944a619c58b2619c87471a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:13.208923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9bd73a68-c927-467e-a6bf-1723f3c9a29f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:13.210522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9bd73a68-c927-467e-a6bf-1723f3c9a29f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:13.219743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 873/1747] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:10:12 2022] DELETE /identity/v3/projects/7b6bf14245944a619c58b2619c87471a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:13.221418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd73a68-c927-467e-a6bf-1723f3c9a29f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:13.224150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9bd73a68-c927-467e-a6bf-1723f3c9a29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:13.299491 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:13.316806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b67ac3-f8aa-48ff-8467-dc4675d615dd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e053b6b25a4e4a20a05e6ffa6683222b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:13.317580 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b67ac3-f8aa-48ff-8467-dc4675d615dd service glance] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:13.317986 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b67ac3-f8aa-48ff-8467-dc4675d615dd service glance] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:13.318342 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b67ac3-f8aa-48ff-8467-dc4675d615dd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:13.321034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d1b67ac3-f8aa-48ff-8467-dc4675d615dd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:13.398201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd73a68-c927-467e-a6bf-1723f3c9a29f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:13.407818 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e83bf4-7a0b-4723-ba66-050a48bae110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:13.409004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e83bf4-7a0b-4723-ba66-050a48bae110 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:13.409920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e83bf4-7a0b-4723-ba66-050a48bae110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:13.410603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e83bf4-7a0b-4723-ba66-050a48bae110 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:13.412117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e83bf4-7a0b-4723-ba66-050a48bae110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:13.415161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e83bf4-7a0b-4723-ba66-050a48bae110 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:13.433020 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 874/1748] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:13.435589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fad5a-b1a1-4a3e-b53a-92298edb94f4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:13.436903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fad5a-b1a1-4a3e-b53a-92298edb94f4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:13.439239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fad5a-b1a1-4a3e-b53a-92298edb94f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:13.446385 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b67ac3-f8aa-48ff-8467-dc4675d615dd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:13.448047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b67ac3-f8aa-48ff-8467-dc4675d615dd service glance] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:13.472222 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 875/1749] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:13.473593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b67ac3-f8aa-48ff-8467-dc4675d615dd service glance] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:13.487868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b67ac3-f8aa-48ff-8467-dc4675d615dd service glance] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:13.493368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ccfa4-d02a-4198-b4e3-67e12b8f4616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:13.494534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ccfa4-d02a-4198-b4e3-67e12b8f4616 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:13.496138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ccfa4-d02a-4198-b4e3-67e12b8f4616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:13.496451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ccfa4-d02a-4198-b4e3-67e12b8f4616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:13.497362 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ccfa4-d02a-4198-b4e3-67e12b8f4616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:13.497858 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-750fad5a-b1a1-4a3e-b53a-92298edb94f4 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:13.502181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ccfa4-d02a-4198-b4e3-67e12b8f4616 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:13.516149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-750fad5a-b1a1-4a3e-b53a-92298edb94f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:13.539167 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 876/1750] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:13.543062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ccfa4-d02a-4198-b4e3-67e12b8f4616 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:13.567756 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ccfa4-d02a-4198-b4e3-67e12b8f4616 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:13.570485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 875/1751] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:13.581753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662ca5a-db49-4be9-afc4-6a66f69812cf None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:13.581980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb3213-e927-4af2-8153-70adbd3b73e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:13.582174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662ca5a-db49-4be9-afc4-6a66f69812cf None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:13.582340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662ca5a-db49-4be9-afc4-6a66f69812cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:13.583174 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb3213-e927-4af2-8153-70adbd3b73e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:13.584240 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb3213-e927-4af2-8153-70adbd3b73e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:13.584240 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb3213-e927-4af2-8153-70adbd3b73e3 admin admin] PATH_INFO: `/v3/projects/3f88472cf646401e83da00606a7daf12/users/0ade1ac3cbbb42dab5a4a8fc36ae2730/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:13.619521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d662ca5a-db49-4be9-afc4-6a66f69812cf None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:13.621455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d662ca5a-db49-4be9-afc4-6a66f69812cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:13.630022 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb3213-e927-4af2-8153-70adbd3b73e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f88472cf646401e83da00606a7daf12, user_id=0ade1ac3cbbb42dab5a4a8fc36ae2730, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:13.640173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb3213-e927-4af2-8153-70adbd3b73e3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:13.649337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 876/1752] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:13.655465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662ca5a-db49-4be9-afc4-6a66f69812cf None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:13.657086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d662ca5a-db49-4be9-afc4-6a66f69812cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:13.682534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 877/1753] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:13 2022] PUT /identity/v3/projects/3f88472cf646401e83da00606a7daf12/users/0ade1ac3cbbb42dab5a4a8fc36ae2730/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:13.686968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb3213-e927-4af2-8153-70adbd3b73e3 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:13.703856 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662ca5a-db49-4be9-afc4-6a66f69812cf None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:13.938147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7925e54-fd0e-4a23-97fd-9c67ca8381d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:13.939299 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7925e54-fd0e-4a23-97fd-9c67ca8381d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:13.939537 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7925e54-fd0e-4a23-97fd-9c67ca8381d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:13.939783 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7925e54-fd0e-4a23-97fd-9c67ca8381d5 admin admin] PATH_INFO: `/v3/projects/d21db19a8b044bd4a224e99976d0e386` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:13.945906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb3213-e927-4af2-8153-70adbd3b73e3 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:13.948843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7925e54-fd0e-4a23-97fd-9c67ca8381d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d21db19a8b044bd4a224e99976d0e386)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:13.959773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe22f4-fb4d-4d26-831d-514e025d9817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:13.960527 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe22f4-fb4d-4d26-831d-514e025d9817 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:13.960838 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe22f4-fb4d-4d26-831d-514e025d9817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:13.960987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7925e54-fd0e-4a23-97fd-9c67ca8381d5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:13.961115 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe22f4-fb4d-4d26-831d-514e025d9817 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:13.961880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe22f4-fb4d-4d26-831d-514e025d9817 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:10:13.962238 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe22f4-fb4d-4d26-831d-514e025d9817 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:13.970330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe22f4-fb4d-4d26-831d-514e025d9817 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:13.977528 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:14.020571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 878/1754] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:10:14.020571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c6fb7-9077-4ba1-a66e-118069e27c78 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:14.020972 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c6fb7-9077-4ba1-a66e-118069e27c78 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:14.021473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c6fb7-9077-4ba1-a66e-118069e27c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:14.092712 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c54c6fb7-9077-4ba1-a66e-118069e27c78 None None] MFA Rules not processed for user `0ade1ac3cbbb42dab5a4a8fc36ae2730`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:14.093984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c54c6fb7-9077-4ba1-a66e-118069e27c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:14.154685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b7925e54-fd0e-4a23-97fd-9c67ca8381d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd21db19a8b044bd4a224e99976d0e386', 'request_id': 'req-b7925e54-fd0e-4a23-97fd-9c67ca8381d5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:14.172650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b7925e54-fd0e-4a23-97fd-9c67ca8381d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd21db19a8b044bd4a224e99976d0e386', 'request_id': 'req-b7925e54-fd0e-4a23-97fd-9c67ca8381d5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:14.195326 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b7925e54-fd0e-4a23-97fd-9c67ca8381d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd21db19a8b044bd4a224e99976d0e386', 'request_id': 'req-b7925e54-fd0e-4a23-97fd-9c67ca8381d5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:14.227298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b7925e54-fd0e-4a23-97fd-9c67ca8381d5 admin admin] The token cache is being invalidate because project d21db19a8b044bd4a224e99976d0e386 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:14.227763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b7925e54-fd0e-4a23-97fd-9c67ca8381d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:14.232477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b7925e54-fd0e-4a23-97fd-9c67ca8381d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:14.235997 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 877/1755] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:10:13 2022] DELETE /identity/v3/projects/d21db19a8b044bd4a224e99976d0e386 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:14.237558 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b6390-ef18-4378-b921-4985bf4cb07d None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:14.237852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b6390-ef18-4378-b921-4985bf4cb07d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:14.238131 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b6390-ef18-4378-b921-4985bf4cb07d None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:14.240760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 878/1756] 10.222.0.13 () {54 vars in 912 bytes} [Thu Jul 21 09:10:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:10:14.248706 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b6390-ef18-4378-b921-4985bf4cb07d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:14.252390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b32b6390-ef18-4378-b921-4985bf4cb07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:14.285548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:14.466260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b6390-ef18-4378-b921-4985bf4cb07d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:14.478231 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63e08b-00f2-47c1-830a-377fd4fb8df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:14.484215 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63e08b-00f2-47c1-830a-377fd4fb8df9 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:14.486973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63e08b-00f2-47c1-830a-377fd4fb8df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:14.486973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63e08b-00f2-47c1-830a-377fd4fb8df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:14.488179 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63e08b-00f2-47c1-830a-377fd4fb8df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:14.494363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63e08b-00f2-47c1-830a-377fd4fb8df9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:14.509028 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 879/1757] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:14.509028 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c6fb7-9077-4ba1-a66e-118069e27c78 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:14.510216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c54c6fb7-9077-4ba1-a66e-118069e27c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:14.548087 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 879/1758] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:14 2022] POST /identity/v3/projects => generated 365 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:14.549818 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487e32c-7e1e-4ab0-913f-1d67e7baee58 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:14.550007 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487e32c-7e1e-4ab0-913f-1d67e7baee58 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:14.550292 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487e32c-7e1e-4ab0-913f-1d67e7baee58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:14.563355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c6fb7-9077-4ba1-a66e-118069e27c78 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:14.705713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-7487e32c-7e1e-4ab0-913f-1d67e7baee58 None None] MFA Rules not processed for user `9e2a13c2cb864175a082de3ca8b166c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:14.713576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7487e32c-7e1e-4ab0-913f-1d67e7baee58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:14.779648 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d08a53-13db-40b2-b3b2-342f7fdae148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:14.782231 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d08a53-13db-40b2-b3b2-342f7fdae148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:14.782736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d08a53-13db-40b2-b3b2-342f7fdae148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:14.783247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d08a53-13db-40b2-b3b2-342f7fdae148 admin admin] PATH_INFO: `/v3/users/86bd1581bdae43b9a7e630aeb2937874` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:14.785893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d08a53-13db-40b2-b3b2-342f7fdae148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86bd1581bdae43b9a7e630aeb2937874)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:14.790002 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d08a53-13db-40b2-b3b2-342f7fdae148 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:14.852747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 880/1759] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jul 21 09:10:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:14.853999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2a515-d900-469f-9a50-c09f3bc75fd8 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:14.854384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2a515-d900-469f-9a50-c09f3bc75fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:14.855360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2a515-d900-469f-9a50-c09f3bc75fd8 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:14.861084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 881/1760] 10.222.0.13 () {54 vars in 912 bytes} [Thu Jul 21 09:10:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:10:14.862193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2a515-d900-469f-9a50-c09f3bc75fd8 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:14.871733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e3f2a515-d900-469f-9a50-c09f3bc75fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:14.908002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f2a515-d900-469f-9a50-c09f3bc75fd8 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:14.924865 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d453d2e-c615-4297-99c8-a3f73778c74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:14.925457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d453d2e-c615-4297-99c8-a3f73778c74b admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:14.925996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d453d2e-c615-4297-99c8-a3f73778c74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:14.926289 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d453d2e-c615-4297-99c8-a3f73778c74b admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:14.927024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d453d2e-c615-4297-99c8-a3f73778c74b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:14.929133 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d453d2e-c615-4297-99c8-a3f73778c74b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:14.932075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-89d08a53-13db-40b2-b3b2-342f7fdae148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86bd1581bdae43b9a7e630aeb2937874'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:14.935157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-89d08a53-13db-40b2-b3b2-342f7fdae148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86bd1581bdae43b9a7e630aeb2937874'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:14.953350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-89d08a53-13db-40b2-b3b2-342f7fdae148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86bd1581bdae43b9a7e630aeb2937874'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:14.957019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:15.005881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 882/1761] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:14 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:15.006941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d453d2e-c615-4297-99c8-a3f73778c74b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:15.011941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0d453d2e-c615-4297-99c8-a3f73778c74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:15.016077 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-89d08a53-13db-40b2-b3b2-342f7fdae148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86bd1581bdae43b9a7e630aeb2937874'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:15.057263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-89d08a53-13db-40b2-b3b2-342f7fdae148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86bd1581bdae43b9a7e630aeb2937874'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:15.085882 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 880/1762] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:10:14 2022] DELETE /identity/v3/users/86bd1581bdae43b9a7e630aeb2937874 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:15.087629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de942a-d96e-4180-abc2-48c3d631cd83 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:15.089835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de942a-d96e-4180-abc2-48c3d631cd83 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:15.090862 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de942a-d96e-4180-abc2-48c3d631cd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:15.153796 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-88de942a-d96e-4180-abc2-48c3d631cd83 None None] MFA Rules not processed for user `9e2a13c2cb864175a082de3ca8b166c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:15.155107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-88de942a-d96e-4180-abc2-48c3d631cd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:15.291270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 881/1763] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jul 21 09:10:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:15.294198 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de942a-d96e-4180-abc2-48c3d631cd83 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:15.300025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-88de942a-d96e-4180-abc2-48c3d631cd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:15.309288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d453d2e-c615-4297-99c8-a3f73778c74b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:15.325611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de942a-d96e-4180-abc2-48c3d631cd83 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:15.334947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28958e0c-9239-4c4a-8d74-51b76467b0ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:15.335724 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28958e0c-9239-4c4a-8d74-51b76467b0ed service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:15.336246 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28958e0c-9239-4c4a-8d74-51b76467b0ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:15.338005 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28958e0c-9239-4c4a-8d74-51b76467b0ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:15.345895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-28958e0c-9239-4c4a-8d74-51b76467b0ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:15.362981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:15.376900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28958e0c-9239-4c4a-8d74-51b76467b0ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:15.379037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28958e0c-9239-4c4a-8d74-51b76467b0ed service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:15.399296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 882/1764] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:15.400489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28958e0c-9239-4c4a-8d74-51b76467b0ed service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:15.406779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-28958e0c-9239-4c4a-8d74-51b76467b0ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:15.561765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303fcc0-8890-44f9-8017-e0a14b1bedae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:15.563075 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303fcc0-8890-44f9-8017-e0a14b1bedae admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:15.563596 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303fcc0-8890-44f9-8017-e0a14b1bedae admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:15.564162 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303fcc0-8890-44f9-8017-e0a14b1bedae admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:15.566775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303fcc0-8890-44f9-8017-e0a14b1bedae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:15.574134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303fcc0-8890-44f9-8017-e0a14b1bedae admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:15.576920 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28958e0c-9239-4c4a-8d74-51b76467b0ed service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:15.583053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4fc59-c324-4ee8-a652-cabf9d354f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:15.584311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4fc59-c324-4ee8-a652-cabf9d354f0b admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:15.584820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4fc59-c324-4ee8-a652-cabf9d354f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:15.585299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4fc59-c324-4ee8-a652-cabf9d354f0b admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:15.586337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4fc59-c324-4ee8-a652-cabf9d354f0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:15.589521 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4fc59-c324-4ee8-a652-cabf9d354f0b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:15.621129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 883/1765] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:15 2022] POST /identity/v3/users => generated 283 bytes in 614 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:15.622738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303fcc0-8890-44f9-8017-e0a14b1bedae admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:15.636575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303fcc0-8890-44f9-8017-e0a14b1bedae admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:15.641538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab7fcb-9d1f-41d8-a2aa-45cdcd935f3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:15.642578 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 883/1766] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:15 2022] POST /identity/v3/users => generated 290 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:15.643104 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab7fcb-9d1f-41d8-a2aa-45cdcd935f3c service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:15.643799 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab7fcb-9d1f-41d8-a2aa-45cdcd935f3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:15.644415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ab7fcb-9d1f-41d8-a2aa-45cdcd935f3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:15.646696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4fc59-c324-4ee8-a652-cabf9d354f0b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:15.657657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab7fcb-9d1f-41d8-a2aa-45cdcd935f3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:15.660038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ab7fcb-9d1f-41d8-a2aa-45cdcd935f3c service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:15.678481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 884/1767] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:15.681594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab7fcb-9d1f-41d8-a2aa-45cdcd935f3c service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:15.681918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4fc59-c324-4ee8-a652-cabf9d354f0b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:15.691096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f78a70-d49b-4154-8389-8c8d9cad13bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:15.692298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f78a70-d49b-4154-8389-8c8d9cad13bb admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:15.692822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f78a70-d49b-4154-8389-8c8d9cad13bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:15.693337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f78a70-d49b-4154-8389-8c8d9cad13bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:15.694630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f78a70-d49b-4154-8389-8c8d9cad13bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:15.701009 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f78a70-d49b-4154-8389-8c8d9cad13bb admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:15.727104 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ab7fcb-9d1f-41d8-a2aa-45cdcd935f3c service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:15.735622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 884/1768] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:15.739773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f78a70-d49b-4154-8389-8c8d9cad13bb admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:15.743836 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3a4c9-2543-451b-a517-53cdcdb8c30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:15.744650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3a4c9-2543-451b-a517-53cdcdb8c30d admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:15.746418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3a4c9-2543-451b-a517-53cdcdb8c30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:15.746418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3a4c9-2543-451b-a517-53cdcdb8c30d admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:15.747091 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3a4c9-2543-451b-a517-53cdcdb8c30d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:15.752984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3a4c9-2543-451b-a517-53cdcdb8c30d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:15.756931 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f78a70-d49b-4154-8389-8c8d9cad13bb admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:15.766286 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069d655-632f-41a7-84af-21bb20ce517a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:15.767293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069d655-632f-41a7-84af-21bb20ce517a admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:15.767845 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069d655-632f-41a7-84af-21bb20ce517a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:15.768354 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069d655-632f-41a7-84af-21bb20ce517a admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:15.770200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e069d655-632f-41a7-84af-21bb20ce517a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:15.772844 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e069d655-632f-41a7-84af-21bb20ce517a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:15.788416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 885/1769] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:15.789849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 885/1770] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:15.793895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3a4c9-2543-451b-a517-53cdcdb8c30d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:15.796884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069d655-632f-41a7-84af-21bb20ce517a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:15.816995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3a4c9-2543-451b-a517-53cdcdb8c30d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:15.824261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d9d48-4ccc-48b0-aeaa-5bd5836bd4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:15.825205 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d9d48-4ccc-48b0-aeaa-5bd5836bd4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:15.825608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d9d48-4ccc-48b0-aeaa-5bd5836bd4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:15.825996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d9d48-4ccc-48b0-aeaa-5bd5836bd4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:15.826756 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069d655-632f-41a7-84af-21bb20ce517a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:15.827042 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d9d48-4ccc-48b0-aeaa-5bd5836bd4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:15.829306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d9d48-4ccc-48b0-aeaa-5bd5836bd4c5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:15.843214 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af1e2b-0d6d-4646-bc47-6bc011cdd952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:15.843707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 886/1771] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:15.844367 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af1e2b-0d6d-4646-bc47-6bc011cdd952 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:15.845158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af1e2b-0d6d-4646-bc47-6bc011cdd952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:15.845158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af1e2b-0d6d-4646-bc47-6bc011cdd952 admin admin] PATH_INFO: `/v3/projects/d16136c7ecd548cdbd1b5d123f21e68f/users/fdb5af6be42848b1aca7be261ae5ae40/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:15.849507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d9d48-4ccc-48b0-aeaa-5bd5836bd4c5 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:15.869494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d9d48-4ccc-48b0-aeaa-5bd5836bd4c5 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:15.886329 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec136140-a722-4be7-9053-0db98e9fdb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:15.889684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec136140-a722-4be7-9053-0db98e9fdb26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:15.892161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec136140-a722-4be7-9053-0db98e9fdb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:15.892935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec136140-a722-4be7-9053-0db98e9fdb26 admin admin] PATH_INFO: `/v3/projects/609b19d2530043a59b76d9b99aebc5b7/users/719adc8fb0bb4be0bfb677c5ce11dd8b/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:15.895127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af1e2b-0d6d-4646-bc47-6bc011cdd952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d16136c7ecd548cdbd1b5d123f21e68f, user_id=fdb5af6be42848b1aca7be261ae5ae40, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:15.898484 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af1e2b-0d6d-4646-bc47-6bc011cdd952 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:15.941267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 886/1772] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:15 2022] PUT /identity/v3/projects/d16136c7ecd548cdbd1b5d123f21e68f/users/fdb5af6be42848b1aca7be261ae5ae40/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:15.951191 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421e488-4b9b-4982-8a5d-219af56eed69 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:15.951376 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec136140-a722-4be7-9053-0db98e9fdb26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=609b19d2530043a59b76d9b99aebc5b7, user_id=719adc8fb0bb4be0bfb677c5ce11dd8b, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:15.951521 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421e488-4b9b-4982-8a5d-219af56eed69 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:15.951772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421e488-4b9b-4982-8a5d-219af56eed69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:15.959555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec136140-a722-4be7-9053-0db98e9fdb26 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:16.026197 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 887/1773] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:15 2022] PUT /identity/v3/projects/609b19d2530043a59b76d9b99aebc5b7/users/719adc8fb0bb4be0bfb677c5ce11dd8b/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:10:16.047969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dfeec-1742-48e0-a28b-bbe63f2a7aae None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:16.047969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dfeec-1742-48e0-a28b-bbe63f2a7aae None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:16.047969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dfeec-1742-48e0-a28b-bbe63f2a7aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:16.048736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f421e488-4b9b-4982-8a5d-219af56eed69 None None] MFA Rules not processed for user `fdb5af6be42848b1aca7be261ae5ae40`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:16.048736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f421e488-4b9b-4982-8a5d-219af56eed69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:16.125251 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-be1dfeec-1742-48e0-a28b-bbe63f2a7aae None None] MFA Rules not processed for user `719adc8fb0bb4be0bfb677c5ce11dd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:16.127633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-be1dfeec-1742-48e0-a28b-bbe63f2a7aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:16.163692 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 887/1774] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:10:16.165377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966f95c-0b7b-4595-aa05-7047a7308379 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:16.168632 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:16.169089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966f95c-0b7b-4595-aa05-7047a7308379 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:16.169621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966f95c-0b7b-4595-aa05-7047a7308379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:16.287980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3966f95c-0b7b-4595-aa05-7047a7308379 None None] MFA Rules not processed for user `e717fb256de24adb8ee6815338a94e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:16.295405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3966f95c-0b7b-4595-aa05-7047a7308379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:16.358984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:16.366986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 888/1775] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:16.366986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dfeec-1742-48e0-a28b-bbe63f2a7aae None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:16.492119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 888/1776] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:16.492119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966f95c-0b7b-4595-aa05-7047a7308379 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:16.549873 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dfeec-1742-48e0-a28b-bbe63f2a7aae None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:16.560338 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf577d16-59e1-411e-97e4-57f5041bf75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:16.562601 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf577d16-59e1-411e-97e4-57f5041bf75c admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:16.563127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf577d16-59e1-411e-97e4-57f5041bf75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:16.564898 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf577d16-59e1-411e-97e4-57f5041bf75c admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:16.565684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf577d16-59e1-411e-97e4-57f5041bf75c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:16.567969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf577d16-59e1-411e-97e4-57f5041bf75c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:16.599588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966f95c-0b7b-4595-aa05-7047a7308379 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:16.606551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1009c9e-191e-4ebf-a187-f981ca6f7dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:16.607656 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1009c9e-191e-4ebf-a187-f981ca6f7dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:16.608199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1009c9e-191e-4ebf-a187-f981ca6f7dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:16.608830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1009c9e-191e-4ebf-a187-f981ca6f7dc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:16.609799 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1009c9e-191e-4ebf-a187-f981ca6f7dc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:16.612802 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1009c9e-191e-4ebf-a187-f981ca6f7dc3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:16.660387 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 889/1777] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:16 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:16.661748 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1e701-45e8-4b99-92d0-f6ee2de37020 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:16.662076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1e701-45e8-4b99-92d0-f6ee2de37020 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:16.662361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1e701-45e8-4b99-92d0-f6ee2de37020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:16.674004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 889/1778] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:16 2022] POST /identity/v3/projects => generated 365 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:16.674896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf577d16-59e1-411e-97e4-57f5041bf75c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:16.690877 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-58c1e701-45e8-4b99-92d0-f6ee2de37020 None None] MFA Rules not processed for user `7b4006c43f7944c083cf323db4df2b3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:16.692251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-58c1e701-45e8-4b99-92d0-f6ee2de37020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:16.861904 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 890/1779] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:16.863376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1e701-45e8-4b99-92d0-f6ee2de37020 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:16.870114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf577d16-59e1-411e-97e4-57f5041bf75c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:16.883983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8d817-267e-49fc-b762-42f50559debf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:16.885136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8d817-267e-49fc-b762-42f50559debf admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:16.885136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8d817-267e-49fc-b762-42f50559debf admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:16.885136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8d817-267e-49fc-b762-42f50559debf admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:16.886552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8d817-267e-49fc-b762-42f50559debf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:16.893098 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8d817-267e-49fc-b762-42f50559debf admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:16.895537 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1e701-45e8-4b99-92d0-f6ee2de37020 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:16.908592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f67b3d-4763-42a1-b22e-aae0a8c32f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:16.909328 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f67b3d-4763-42a1-b22e-aae0a8c32f0a admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:16.909667 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f67b3d-4763-42a1-b22e-aae0a8c32f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:16.909907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f67b3d-4763-42a1-b22e-aae0a8c32f0a admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:16.910957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f67b3d-4763-42a1-b22e-aae0a8c32f0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:16.913047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f67b3d-4763-42a1-b22e-aae0a8c32f0a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:16.960623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 891/1780] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:16 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:16.967884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f67b3d-4763-42a1-b22e-aae0a8c32f0a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:16.994656 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f67b3d-4763-42a1-b22e-aae0a8c32f0a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:17.008014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 890/1781] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:16 2022] POST /identity/v3/users => generated 296 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:17.010381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3172e-6715-40b5-a8b9-d7c3df16d4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:17.011723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3172e-6715-40b5-a8b9-d7c3df16d4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:17.011956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3172e-6715-40b5-a8b9-d7c3df16d4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:17.012948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3172e-6715-40b5-a8b9-d7c3df16d4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:17.015344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8d817-267e-49fc-b762-42f50559debf admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:17.015510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3172e-6715-40b5-a8b9-d7c3df16d4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:17.023538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3172e-6715-40b5-a8b9-d7c3df16d4b7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:17.055105 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8d817-267e-49fc-b762-42f50559debf admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:17.071549 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 892/1782] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:17.074051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3172e-6715-40b5-a8b9-d7c3df16d4b7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:17.077839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d6e3172e-6715-40b5-a8b9-d7c3df16d4b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:17.097392 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca012c-d8e6-4cb4-91f0-d04835eb8734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:17.098728 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca012c-d8e6-4cb4-91f0-d04835eb8734 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:17.100932 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca012c-d8e6-4cb4-91f0-d04835eb8734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:17.101469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca012c-d8e6-4cb4-91f0-d04835eb8734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:17.110312 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca012c-d8e6-4cb4-91f0-d04835eb8734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:17.118933 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca012c-d8e6-4cb4-91f0-d04835eb8734 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:17.171319 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:17.250122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 891/1783] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:17.259158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca012c-d8e6-4cb4-91f0-d04835eb8734 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:17.332426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca012c-d8e6-4cb4-91f0-d04835eb8734 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:17.381402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f65c9-db5d-4b2a-a41e-134a2bc309aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:17.387844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f65c9-db5d-4b2a-a41e-134a2bc309aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:17.388957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f65c9-db5d-4b2a-a41e-134a2bc309aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:17.396059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f65c9-db5d-4b2a-a41e-134a2bc309aa admin admin] PATH_INFO: `/v3/projects/5a06ead90e4f4477afbcf78794ac38b2/users/53df719c21dc4049a1292f33dfdcef78/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:17.445717 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3172e-6715-40b5-a8b9-d7c3df16d4b7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:17.469830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35a261-d039-40c1-ba86-18ee0536115c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:17.471498 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35a261-d039-40c1-ba86-18ee0536115c service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:17.475436 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35a261-d039-40c1-ba86-18ee0536115c service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:17.475902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35a261-d039-40c1-ba86-18ee0536115c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:17.484232 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f65c9-db5d-4b2a-a41e-134a2bc309aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a06ead90e4f4477afbcf78794ac38b2, user_id=53df719c21dc4049a1292f33dfdcef78, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:17.489349 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f65c9-db5d-4b2a-a41e-134a2bc309aa admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:17.516974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:17.534217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 892/1784] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:17 2022] PUT /identity/v3/projects/5a06ead90e4f4477afbcf78794ac38b2/users/53df719c21dc4049a1292f33dfdcef78/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:17.535085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f65c9-db5d-4b2a-a41e-134a2bc309aa admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:17.681724 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a35a261-d039-40c1-ba86-18ee0536115c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:17.686388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a35a261-d039-40c1-ba86-18ee0536115c service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:17.708597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f65c9-db5d-4b2a-a41e-134a2bc309aa admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:17.722571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 893/1785] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:17.724209 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2472a-2df0-4fc1-ad7c-ab2703f5c69d None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:17.724502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2472a-2df0-4fc1-ad7c-ab2703f5c69d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:17.724779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2472a-2df0-4fc1-ad7c-ab2703f5c69d None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:17.725638 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc1f6a-729f-4eb4-b488-375293b62512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:17.727832 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc1f6a-729f-4eb4-b488-375293b62512 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:17.729078 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc1f6a-729f-4eb4-b488-375293b62512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:17.729819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc1f6a-729f-4eb4-b488-375293b62512 admin admin] PATH_INFO: `/v3/projects/564af1dfb6f9439aab14191a374130b0/users/ef1182541cd04cdd88d650d9b7846aed/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:17.731199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 894/1786] 10.222.0.13 () {54 vars in 961 bytes} [Thu Jul 21 09:10:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:10:17.732196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2472a-2df0-4fc1-ad7c-ab2703f5c69d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:17.737008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-97c2472a-2df0-4fc1-ad7c-ab2703f5c69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:17.767881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2472a-2df0-4fc1-ad7c-ab2703f5c69d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:17.781350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b09d29-94d1-4d45-9326-fd49a9172f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:17.782557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b09d29-94d1-4d45-9326-fd49a9172f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:17.783731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b09d29-94d1-4d45-9326-fd49a9172f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:17.785728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b09d29-94d1-4d45-9326-fd49a9172f8e admin admin] PATH_INFO: `/v3/projects/871235f3f2e0474d9f57df38dba4fcf2` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:17.799611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b09d29-94d1-4d45-9326-fd49a9172f8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=871235f3f2e0474d9f57df38dba4fcf2)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:17.802584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b09d29-94d1-4d45-9326-fd49a9172f8e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:17.819206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc1f6a-729f-4eb4-b488-375293b62512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=564af1dfb6f9439aab14191a374130b0, user_id=ef1182541cd04cdd88d650d9b7846aed, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:17.840800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc1f6a-729f-4eb4-b488-375293b62512 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:17.917654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 893/1787] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:17 2022] PUT /identity/v3/projects/564af1dfb6f9439aab14191a374130b0/users/ef1182541cd04cdd88d650d9b7846aed/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:17.919114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc1f6a-729f-4eb4-b488-375293b62512 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:17.922202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-93dc1f6a-729f-4eb4-b488-375293b62512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:17.944583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a2b09d29-94d1-4d45-9326-fd49a9172f8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '871235f3f2e0474d9f57df38dba4fcf2', 'request_id': 'req-a2b09d29-94d1-4d45-9326-fd49a9172f8e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:17.968066 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a2b09d29-94d1-4d45-9326-fd49a9172f8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '871235f3f2e0474d9f57df38dba4fcf2', 'request_id': 'req-a2b09d29-94d1-4d45-9326-fd49a9172f8e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:17.998779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a2b09d29-94d1-4d45-9326-fd49a9172f8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '871235f3f2e0474d9f57df38dba4fcf2', 'request_id': 'req-a2b09d29-94d1-4d45-9326-fd49a9172f8e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:18.045946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a2b09d29-94d1-4d45-9326-fd49a9172f8e admin admin] The token cache is being invalidate because project 871235f3f2e0474d9f57df38dba4fcf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:18.046246 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a2b09d29-94d1-4d45-9326-fd49a9172f8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:18.047271 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a2b09d29-94d1-4d45-9326-fd49a9172f8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:18.051276 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 895/1788] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:10:17 2022] DELETE /identity/v3/projects/871235f3f2e0474d9f57df38dba4fcf2 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:18.053438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b09d29-94d1-4d45-9326-fd49a9172f8e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:18.089491 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a2b09d29-94d1-4d45-9326-fd49a9172f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:18.170815 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc1f6a-729f-4eb4-b488-375293b62512 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:18.329318 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:18.493060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b09d29-94d1-4d45-9326-fd49a9172f8e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:18.506715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5be04c-9694-4f23-8e2f-1feeb58d64a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:18.507706 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5be04c-9694-4f23-8e2f-1feeb58d64a1 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:18.508213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5be04c-9694-4f23-8e2f-1feeb58d64a1 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:18.511425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5be04c-9694-4f23-8e2f-1feeb58d64a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:18.516974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6e5be04c-9694-4f23-8e2f-1feeb58d64a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:18.533872 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25cade-b53f-49bf-ab8f-a17cf18f30fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:18.534729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25cade-b53f-49bf-ab8f-a17cf18f30fa admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:18.535239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25cade-b53f-49bf-ab8f-a17cf18f30fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:18.535811 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25cade-b53f-49bf-ab8f-a17cf18f30fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:18.537316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25cade-b53f-49bf-ab8f-a17cf18f30fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:18.549672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5be04c-9694-4f23-8e2f-1feeb58d64a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:18.551798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25cade-b53f-49bf-ab8f-a17cf18f30fa admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:18.552420 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5be04c-9694-4f23-8e2f-1feeb58d64a1 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:18.606841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 894/1789] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:18.612360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42bc9c-1e41-4de4-bd2c-9f61b90d1a19 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:18.612638 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42bc9c-1e41-4de4-bd2c-9f61b90d1a19 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:18.612929 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42bc9c-1e41-4de4-bd2c-9f61b90d1a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:18.614320 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 896/1790] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 562 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:18.615112 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25cade-b53f-49bf-ab8f-a17cf18f30fa admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:18.637245 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fb25cade-b53f-49bf-ab8f-a17cf18f30fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:18.703135 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5a42bc9c-1e41-4de4-bd2c-9f61b90d1a19 None None] MFA Rules not processed for user `72f9ef6de66e456e800237e9f954e98d`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:18.709894 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5a42bc9c-1e41-4de4-bd2c-9f61b90d1a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:18.719544 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25cade-b53f-49bf-ab8f-a17cf18f30fa admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:18.740486 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35787c62-06d3-4df9-8ec9-12af47d4a9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:18.741212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35787c62-06d3-4df9-8ec9-12af47d4a9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:18.741499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35787c62-06d3-4df9-8ec9-12af47d4a9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:18.741783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35787c62-06d3-4df9-8ec9-12af47d4a9b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:18.742543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35787c62-06d3-4df9-8ec9-12af47d4a9b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:18.744289 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35787c62-06d3-4df9-8ec9-12af47d4a9b3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:18.749556 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 895/1791] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Jul 21 09:10:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:18.750942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0ccd2-29a3-49f9-9ecc-611098049778 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:18.751236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0ccd2-29a3-49f9-9ecc-611098049778 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:18.751518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0ccd2-29a3-49f9-9ecc-611098049778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:18.795960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 897/1792] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:18.796796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:18.799102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3dd0a-1a6a-42f4-8116-0f3cc504b174 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:18.801839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3dd0a-1a6a-42f4-8116-0f3cc504b174 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:18.802988 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3dd0a-1a6a-42f4-8116-0f3cc504b174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:18.821466 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-14f0ccd2-29a3-49f9-9ecc-611098049778 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:18.823821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-14f0ccd2-29a3-49f9-9ecc-611098049778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:18.845949 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 896/1793] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:18.846812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0ccd2-29a3-49f9-9ecc-611098049778 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:18.871409 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0ccd2-29a3-49f9-9ecc-611098049778 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:18.882173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db57a5ed-d4a2-4977-aa73-3e15115ed52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:18.883109 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57a5ed-d4a2-4977-aa73-3e15115ed52b admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:18.883395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57a5ed-d4a2-4977-aa73-3e15115ed52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:18.883675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57a5ed-d4a2-4977-aa73-3e15115ed52b admin admin] PATH_INFO: `/v3/projects/5a06ead90e4f4477afbcf78794ac38b2/users/53df719c21dc4049a1292f33dfdcef78/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:18.897483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db57a5ed-d4a2-4977-aa73-3e15115ed52b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a06ead90e4f4477afbcf78794ac38b2, user_id=53df719c21dc4049a1292f33dfdcef78, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:18.901418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db57a5ed-d4a2-4977-aa73-3e15115ed52b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:18.926910 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 897/1794] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:18 2022] PUT /identity/v3/projects/5a06ead90e4f4477afbcf78794ac38b2/users/53df719c21dc4049a1292f33dfdcef78/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:18.929949 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e9e3dd0a-1a6a-42f4-8116-0f3cc504b174 None None] MFA Rules not processed for user `6c62db4b38434cfe886108a72e4c42aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:18.936964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e9e3dd0a-1a6a-42f4-8116-0f3cc504b174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:18.938418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52384a-0644-4a47-9142-6e1ab10275f8 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:18.938664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52384a-0644-4a47-9142-6e1ab10275f8 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:18.938949 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52384a-0644-4a47-9142-6e1ab10275f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:18.983270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4d52384a-0644-4a47-9142-6e1ab10275f8 None None] MFA Rules not processed for user `be1372c20ad94145b775663a9037d55a`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:18.984701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4d52384a-0644-4a47-9142-6e1ab10275f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:19.176070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 898/1795] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:19.195636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52384a-0644-4a47-9142-6e1ab10275f8 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:19.274292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 898/1796] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:19.274292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3dd0a-1a6a-42f4-8116-0f3cc504b174 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:19.536811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52384a-0644-4a47-9142-6e1ab10275f8 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:19.563525 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4f176-abd7-4748-bef8-fddf8b2f7260 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:19.564835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4f176-abd7-4748-bef8-fddf8b2f7260 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:19.564835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4f176-abd7-4748-bef8-fddf8b2f7260 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:19.567112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4f176-abd7-4748-bef8-fddf8b2f7260 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:19.590782 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4f176-abd7-4748-bef8-fddf8b2f7260 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:19.594003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4f176-abd7-4748-bef8-fddf8b2f7260 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:19.647316 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 899/1797] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:19.649879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463531a-c4fe-451b-80c2-7d6f19c68685 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:19.650175 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463531a-c4fe-451b-80c2-7d6f19c68685 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:19.650455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463531a-c4fe-451b-80c2-7d6f19c68685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:19.669735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3dd0a-1a6a-42f4-8116-0f3cc504b174 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:19.696653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94e80e-5f28-4ca8-8318-bfb0bf96e846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:19.702489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94e80e-5f28-4ca8-8318-bfb0bf96e846 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:19.704834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94e80e-5f28-4ca8-8318-bfb0bf96e846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:19.706195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94e80e-5f28-4ca8-8318-bfb0bf96e846 admin admin] PATH_INFO: `/v3/projects/564af1dfb6f9439aab14191a374130b0/users/ef1182541cd04cdd88d650d9b7846aed/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:19.727417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94e80e-5f28-4ca8-8318-bfb0bf96e846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=564af1dfb6f9439aab14191a374130b0, user_id=ef1182541cd04cdd88d650d9b7846aed, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:19.754715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e463531a-c4fe-451b-80c2-7d6f19c68685 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:19.759643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e463531a-c4fe-451b-80c2-7d6f19c68685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:19.764563 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94e80e-5f28-4ca8-8318-bfb0bf96e846 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:19.792822 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 900/1798] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:19.801468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34125eb1-9548-4a13-8de6-dc859e32d39f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:19.802044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34125eb1-9548-4a13-8de6-dc859e32d39f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:19.802581 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34125eb1-9548-4a13-8de6-dc859e32d39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:19.856787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 899/1799] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:19 2022] PUT /identity/v3/projects/564af1dfb6f9439aab14191a374130b0/users/ef1182541cd04cdd88d650d9b7846aed/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:19.871616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94e80e-5f28-4ca8-8318-bfb0bf96e846 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:19.902365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-34125eb1-9548-4a13-8de6-dc859e32d39f None None] MFA Rules not processed for user `53df719c21dc4049a1292f33dfdcef78`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:19.910151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-34125eb1-9548-4a13-8de6-dc859e32d39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:20.065327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:20.116346 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94e80e-5f28-4ca8-8318-bfb0bf96e846 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:20.134905 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed18e2f-5a9d-408b-b9e0-1df9da19afd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:20.136261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed18e2f-5a9d-408b-b9e0-1df9da19afd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:20.136556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed18e2f-5a9d-408b-b9e0-1df9da19afd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:20.136836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed18e2f-5a9d-408b-b9e0-1df9da19afd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:20.137668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed18e2f-5a9d-408b-b9e0-1df9da19afd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:10:20.138790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed18e2f-5a9d-408b-b9e0-1df9da19afd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:20.142323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed18e2f-5a9d-408b-b9e0-1df9da19afd8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:20.198982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 900/1800] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:10:20.203063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4d0d0-1cc1-4c83-9674-526f36bd3553 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:20.204166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4d0d0-1cc1-4c83-9674-526f36bd3553 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:20.204166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4d0d0-1cc1-4c83-9674-526f36bd3553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:20.243752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:20.257576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 901/1801] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:19 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:20.259538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34125eb1-9548-4a13-8de6-dc859e32d39f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:20.284240 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-09c4d0d0-1cc1-4c83-9674-526f36bd3553 None None] MFA Rules not processed for user `ef1182541cd04cdd88d650d9b7846aed`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:20.285599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-09c4d0d0-1cc1-4c83-9674-526f36bd3553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:20.333929 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34125eb1-9548-4a13-8de6-dc859e32d39f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:20.360650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78218b-3c11-463c-b4ec-d8473be68f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:20.368294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78218b-3c11-463c-b4ec-d8473be68f38 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:20.379458 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78218b-3c11-463c-b4ec-d8473be68f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:20.386173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78218b-3c11-463c-b4ec-d8473be68f38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:20.386173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78218b-3c11-463c-b4ec-d8473be68f38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:20.386173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78218b-3c11-463c-b4ec-d8473be68f38 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:20.471368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 902/1802] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:20 2022] POST /identity/v3/projects => generated 379 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:20.472720 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78218b-3c11-463c-b4ec-d8473be68f38 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:20.479031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3b78218b-3c11-463c-b4ec-d8473be68f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:20.757954 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78218b-3c11-463c-b4ec-d8473be68f38 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:20.766102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 901/1803] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 568 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:20.767654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4d0d0-1cc1-4c83-9674-526f36bd3553 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:20.783098 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f66ad-aafc-4c0d-bebb-55ba2175b0f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:20.783840 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f66ad-aafc-4c0d-bebb-55ba2175b0f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:20.788386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f66ad-aafc-4c0d-bebb-55ba2175b0f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:20.788721 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f66ad-aafc-4c0d-bebb-55ba2175b0f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:20.823984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4d0d0-1cc1-4c83-9674-526f36bd3553 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:21.002312 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f66ad-aafc-4c0d-bebb-55ba2175b0f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:21.003832 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f66ad-aafc-4c0d-bebb-55ba2175b0f6 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:21.038732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 903/1804] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:20 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:21.039864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f66ad-aafc-4c0d-bebb-55ba2175b0f6 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:21.043308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ff1f66ad-aafc-4c0d-bebb-55ba2175b0f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:21.121333 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efabfcc-9158-4416-b751-c693feef9a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:21.122757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efabfcc-9158-4416-b751-c693feef9a97 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:21.122757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efabfcc-9158-4416-b751-c693feef9a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:21.122757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efabfcc-9158-4416-b751-c693feef9a97 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:21.124049 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efabfcc-9158-4416-b751-c693feef9a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:21.128185 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efabfcc-9158-4416-b751-c693feef9a97 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:21.181931 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 902/1805] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:20 2022] POST /identity/v3/users => generated 290 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:21.193275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efabfcc-9158-4416-b751-c693feef9a97 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:21.196451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8efabfcc-9158-4416-b751-c693feef9a97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:21.228294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efabfcc-9158-4416-b751-c693feef9a97 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:21.243422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898e3f0-40b7-4bf9-aa85-4a34920a7781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:21.244147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898e3f0-40b7-4bf9-aa85-4a34920a7781 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:21.244436 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898e3f0-40b7-4bf9-aa85-4a34920a7781 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:21.247875 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898e3f0-40b7-4bf9-aa85-4a34920a7781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:21.277217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f66ad-aafc-4c0d-bebb-55ba2175b0f6 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:21.295455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3490f-7d07-4bb3-8385-dd215ce32610 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:21.301336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3490f-7d07-4bb3-8385-dd215ce32610 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:21.302645 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3490f-7d07-4bb3-8385-dd215ce32610 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:21.304165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3490f-7d07-4bb3-8385-dd215ce32610 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:21.435989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:21.471708 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:21.509282 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898e3f0-40b7-4bf9-aa85-4a34920a7781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:21.512996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898e3f0-40b7-4bf9-aa85-4a34920a7781 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:21.556649 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 903/1806] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:21.559135 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898e3f0-40b7-4bf9-aa85-4a34920a7781 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:21.602343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898e3f0-40b7-4bf9-aa85-4a34920a7781 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:21.625696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3490f-7d07-4bb3-8385-dd215ce32610 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:21.633164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf04e2-010f-4720-8299-df3cdf5866ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:21.633164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf04e2-010f-4720-8299-df3cdf5866ae admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:21.633465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3490f-7d07-4bb3-8385-dd215ce32610 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:21.633907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf04e2-010f-4720-8299-df3cdf5866ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:21.634529 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf04e2-010f-4720-8299-df3cdf5866ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:21.640018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf04e2-010f-4720-8299-df3cdf5866ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:21.652558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf04e2-010f-4720-8299-df3cdf5866ae admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:21.715039 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 904/1807] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 670 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:21.715039 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3490f-7d07-4bb3-8385-dd215ce32610 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:21.736968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 904/1808] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:21.739136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf04e2-010f-4720-8299-df3cdf5866ae admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:21.763652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3490f-7d07-4bb3-8385-dd215ce32610 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:21.783345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6214d-110a-4c40-b3f6-e225e596b84d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:21.786156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf04e2-010f-4720-8299-df3cdf5866ae admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:21.786860 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6214d-110a-4c40-b3f6-e225e596b84d service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:21.787526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6214d-110a-4c40-b3f6-e225e596b84d service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:21.788051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6214d-110a-4c40-b3f6-e225e596b84d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:21.795076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f5b6214d-110a-4c40-b3f6-e225e596b84d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:21.805520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecb0c2-91ba-48fe-a21e-f2c2db79bf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:21.808509 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecb0c2-91ba-48fe-a21e-f2c2db79bf1b admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:21.809077 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecb0c2-91ba-48fe-a21e-f2c2db79bf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:21.809570 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecb0c2-91ba-48fe-a21e-f2c2db79bf1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:21.811548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aecb0c2-91ba-48fe-a21e-f2c2db79bf1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:21.820920 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aecb0c2-91ba-48fe-a21e-f2c2db79bf1b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:21.886905 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 905/1809] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:21.888965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecb0c2-91ba-48fe-a21e-f2c2db79bf1b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:21.951962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecb0c2-91ba-48fe-a21e-f2c2db79bf1b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:21.981853 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e119887-a6d0-488d-aab7-0fa00fd040af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:21.991966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e119887-a6d0-488d-aab7-0fa00fd040af service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:21.991966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e119887-a6d0-488d-aab7-0fa00fd040af service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:21.998139 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e119887-a6d0-488d-aab7-0fa00fd040af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:22.007554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0e119887-a6d0-488d-aab7-0fa00fd040af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:22.091857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6214d-110a-4c40-b3f6-e225e596b84d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:22.096810 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6214d-110a-4c40-b3f6-e225e596b84d service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:22.123604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 905/1810] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:21 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:22.125425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6214d-110a-4c40-b3f6-e225e596b84d service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:22.173390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6214d-110a-4c40-b3f6-e225e596b84d service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:22.197411 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6199e-fd9e-43ff-b314-279c9fe3cef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:22.202261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6199e-fd9e-43ff-b314-279c9fe3cef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:22.204062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6199e-fd9e-43ff-b314-279c9fe3cef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:22.206289 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6199e-fd9e-43ff-b314-279c9fe3cef4 admin admin] PATH_INFO: `/v3/projects/fb724d1f2a5d4154a1ca2f406a11653f/users/ee012e49d4394539bd68d05305a6b7ea/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:22.212881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e119887-a6d0-488d-aab7-0fa00fd040af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:22.216919 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e119887-a6d0-488d-aab7-0fa00fd040af service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:22.264057 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 906/1811] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:22.336730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b6199e-fd9e-43ff-b314-279c9fe3cef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb724d1f2a5d4154a1ca2f406a11653f, user_id=ee012e49d4394539bd68d05305a6b7ea, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:22.358335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b6199e-fd9e-43ff-b314-279c9fe3cef4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:22.455524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 906/1812] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:22 2022] PUT /identity/v3/projects/fb724d1f2a5d4154a1ca2f406a11653f/users/ee012e49d4394539bd68d05305a6b7ea/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:22.478760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a6d76-eaf7-456e-b088-cb586b032300 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:22.480259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a6d76-eaf7-456e-b088-cb586b032300 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:22.480932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a6d76-eaf7-456e-b088-cb586b032300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:22.656687 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-eb9a6d76-eaf7-456e-b088-cb586b032300 None None] MFA Rules not processed for user `ee012e49d4394539bd68d05305a6b7ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:22.674998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-eb9a6d76-eaf7-456e-b088-cb586b032300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:23.121824 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6199e-fd9e-43ff-b314-279c9fe3cef4 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:23.122374 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 907/1813] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 647 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:23.469777 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6199e-fd9e-43ff-b314-279c9fe3cef4 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:23.513336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a666a2-991b-4327-9b21-46b131cbac37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:23.514336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a666a2-991b-4327-9b21-46b131cbac37 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:23.514919 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a666a2-991b-4327-9b21-46b131cbac37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:23.515293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a666a2-991b-4327-9b21-46b131cbac37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:23.516634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a666a2-991b-4327-9b21-46b131cbac37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:23.570357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a666a2-991b-4327-9b21-46b131cbac37 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:23.818697 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 907/1814] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:23 2022] POST /identity/v3/projects => generated 379 bytes in 699 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:10:23.822605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a6d76-eaf7-456e-b088-cb586b032300 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:23.848198 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a6d76-eaf7-456e-b088-cb586b032300 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:24.308920 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:24.385008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68cfc6-d68b-4e15-8507-54d77f4c3115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:24.386718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68cfc6-d68b-4e15-8507-54d77f4c3115 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:24.387213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68cfc6-d68b-4e15-8507-54d77f4c3115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:24.387700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68cfc6-d68b-4e15-8507-54d77f4c3115 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:24.389029 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68cfc6-d68b-4e15-8507-54d77f4c3115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:24.423198 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68cfc6-d68b-4e15-8507-54d77f4c3115 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:24.445685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a8d26-288d-4e4c-9af3-a8195aac6609 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:24.451962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a8d26-288d-4e4c-9af3-a8195aac6609 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:24.452758 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a8d26-288d-4e4c-9af3-a8195aac6609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:24.557969 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 908/1815] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:23 2022] POST /identity/v3/users => generated 296 bytes in 736 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:24.567072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:24.582940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68cfc6-d68b-4e15-8507-54d77f4c3115 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:24.654717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c57a8d26-288d-4e4c-9af3-a8195aac6609 None None] MFA Rules not processed for user `257450db238d4efda783e30cbcd23d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:24.666090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c57a8d26-288d-4e4c-9af3-a8195aac6609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:24.726786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68cfc6-d68b-4e15-8507-54d77f4c3115 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:24.778862 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99a7cc-0961-4f96-b79c-b77d24219e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:24.784420 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99a7cc-0961-4f96-b79c-b77d24219e80 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:24.786160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99a7cc-0961-4f96-b79c-b77d24219e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:24.791349 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99a7cc-0961-4f96-b79c-b77d24219e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:24.799157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99a7cc-0961-4f96-b79c-b77d24219e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:24.818093 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99a7cc-0961-4f96-b79c-b77d24219e80 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:24.976608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 909/1816] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:24.979216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99a7cc-0961-4f96-b79c-b77d24219e80 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:25.038842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 908/1817] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jul 21 09:10:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 594 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:10:25.054728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99a7cc-0961-4f96-b79c-b77d24219e80 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:25.073427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9761ed-463f-4c36-aa2d-90a1a57dd47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:25.075184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9761ed-463f-4c36-aa2d-90a1a57dd47b admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:25.075892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9761ed-463f-4c36-aa2d-90a1a57dd47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:25.076646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9761ed-463f-4c36-aa2d-90a1a57dd47b admin admin] PATH_INFO: `/v3/projects/b0a5e3e895e74c8f91f9a6176a431df0/users/be775a6745ad46cd94aa223d824a0429/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:25.180433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9761ed-463f-4c36-aa2d-90a1a57dd47b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0a5e3e895e74c8f91f9a6176a431df0, user_id=be775a6745ad46cd94aa223d824a0429, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:25.190474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9761ed-463f-4c36-aa2d-90a1a57dd47b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:25.234181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 910/1818] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:24 2022] PUT /identity/v3/projects/b0a5e3e895e74c8f91f9a6176a431df0/users/be775a6745ad46cd94aa223d824a0429/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:25.239798 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a8d26-288d-4e4c-9af3-a8195aac6609 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:25.335081 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a8d26-288d-4e4c-9af3-a8195aac6609 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:25.410815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5968aa-dd2d-45be-8922-9d8b320b4eb7 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:25.412895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5968aa-dd2d-45be-8922-9d8b320b4eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:25.413733 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5968aa-dd2d-45be-8922-9d8b320b4eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:25.468386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7d5968aa-dd2d-45be-8922-9d8b320b4eb7 None None] MFA Rules not processed for user `e717fb256de24adb8ee6815338a94e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:25.474152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7d5968aa-dd2d-45be-8922-9d8b320b4eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:25.724409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 911/1819] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:25.724409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5968aa-dd2d-45be-8922-9d8b320b4eb7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:25.724409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7d5968aa-dd2d-45be-8922-9d8b320b4eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:25.793710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0bb6b-188d-4e89-b081-95baa09f5007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:25.795412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0bb6b-188d-4e89-b081-95baa09f5007 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:25.797004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0bb6b-188d-4e89-b081-95baa09f5007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:25.798982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0bb6b-188d-4e89-b081-95baa09f5007 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:25.805616 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea0bb6b-188d-4e89-b081-95baa09f5007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:25.821558 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea0bb6b-188d-4e89-b081-95baa09f5007 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:25.893751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 909/1820] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 654 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:25.896801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0bb6b-188d-4e89-b081-95baa09f5007 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:26.065989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0bb6b-188d-4e89-b081-95baa09f5007 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:26.119285 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617403f-0550-45c0-b5f8-f631b4b9b9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:26.121088 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617403f-0550-45c0-b5f8-f631b4b9b9ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:26.121499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617403f-0550-45c0-b5f8-f631b4b9b9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:26.121789 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617403f-0550-45c0-b5f8-f631b4b9b9ab admin admin] PATH_INFO: `/v3/projects/b0a5e3e895e74c8f91f9a6176a431df0/users/be775a6745ad46cd94aa223d824a0429/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:26.142673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617403f-0550-45c0-b5f8-f631b4b9b9ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0a5e3e895e74c8f91f9a6176a431df0, user_id=be775a6745ad46cd94aa223d824a0429, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:26.145703 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617403f-0550-45c0-b5f8-f631b4b9b9ab admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:26.206849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 910/1821] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:25 2022] PUT /identity/v3/projects/b0a5e3e895e74c8f91f9a6176a431df0/users/be775a6745ad46cd94aa223d824a0429/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:26.224882 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e8ced-282e-42e7-bd27-38ba04ee013a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:26.225189 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e8ced-282e-42e7-bd27-38ba04ee013a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:26.225446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e8ced-282e-42e7-bd27-38ba04ee013a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:26.267465 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5968aa-dd2d-45be-8922-9d8b320b4eb7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:26.348724 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-aa3e8ced-282e-42e7-bd27-38ba04ee013a None None] MFA Rules not processed for user `be775a6745ad46cd94aa223d824a0429`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:26.350136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-aa3e8ced-282e-42e7-bd27-38ba04ee013a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:26.441773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:26.553866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e8d45-4ef2-4c7c-9465-8469f7d33304 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061def74ec7f42ce9a174e0944bf499f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:26.558118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e8d45-4ef2-4c7c-9465-8469f7d33304 service swift] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:26.559953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e8d45-4ef2-4c7c-9465-8469f7d33304 service swift] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:26.561299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e8d45-4ef2-4c7c-9465-8469f7d33304 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:26.622713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 911/1822] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:26.637161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e8ced-282e-42e7-bd27-38ba04ee013a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:26.650217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:26.929881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e8d45-4ef2-4c7c-9465-8469f7d33304 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:26.932275 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e8ced-282e-42e7-bd27-38ba04ee013a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:26.943146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e8d45-4ef2-4c7c-9465-8469f7d33304 service swift] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:26.976950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbaeeb8-76af-4980-ae95-678d1557c1b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:26.978746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaeeb8-76af-4980-ae95-678d1557c1b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:26.982245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaeeb8-76af-4980-ae95-678d1557c1b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:26.983544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbaeeb8-76af-4980-ae95-678d1557c1b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:27.005992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 912/1823] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jul 21 09:10:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1285 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 09:10:27.034266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbaeeb8-76af-4980-ae95-678d1557c1b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:27.037568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbaeeb8-76af-4980-ae95-678d1557c1b0 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:27.094499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 912/1824] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:28.391683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e8d45-4ef2-4c7c-9465-8469f7d33304 service swift] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:28.542283 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbaeeb8-76af-4980-ae95-678d1557c1b0 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:28.809174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbaeeb8-76af-4980-ae95-678d1557c1b0 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:28.819698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5b244-3f69-4426-bede-6505c0a158d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:28.820721 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5b244-3f69-4426-bede-6505c0a158d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:28.821091 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5b244-3f69-4426-bede-6505c0a158d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:28.821462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5b244-3f69-4426-bede-6505c0a158d8 admin admin] PATH_INFO: `/v3/users/0ade1ac3cbbb42dab5a4a8fc36ae2730` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:28.823616 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5b244-3f69-4426-bede-6505c0a158d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ade1ac3cbbb42dab5a4a8fc36ae2730)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:28.826010 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5b244-3f69-4426-bede-6505c0a158d8 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:28.874640 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e8d45-4ef2-4c7c-9465-8469f7d33304 service swift] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:28.888518 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7094d6f-1d00-4c57-a77b-fe3494b69f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:28.889685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7094d6f-1d00-4c57-a77b-fe3494b69f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:28.890055 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7094d6f-1d00-4c57-a77b-fe3494b69f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:28.902914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7094d6f-1d00-4c57-a77b-fe3494b69f3b admin admin] PATH_INFO: `/v3/users/ee012e49d4394539bd68d05305a6b7ea` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:28.907518 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7094d6f-1d00-4c57-a77b-fe3494b69f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee012e49d4394539bd68d05305a6b7ea)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:28.913212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7094d6f-1d00-4c57-a77b-fe3494b69f3b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:28.935882 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-12b5b244-3f69-4426-bede-6505c0a158d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ade1ac3cbbb42dab5a4a8fc36ae2730'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:28.972722 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-12b5b244-3f69-4426-bede-6505c0a158d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ade1ac3cbbb42dab5a4a8fc36ae2730'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:28.993162 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-12b5b244-3f69-4426-bede-6505c0a158d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ade1ac3cbbb42dab5a4a8fc36ae2730'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:29.046219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-12b5b244-3f69-4426-bede-6505c0a158d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ade1ac3cbbb42dab5a4a8fc36ae2730'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:29.052190 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-12b5b244-3f69-4426-bede-6505c0a158d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ade1ac3cbbb42dab5a4a8fc36ae2730'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:29.079092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 913/1825] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:10:28 2022] DELETE /identity/v3/users/0ade1ac3cbbb42dab5a4a8fc36ae2730 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:29.094274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9f486-0aac-4575-a0e1-a454fa3c12ee None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:29.095350 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9f486-0aac-4575-a0e1-a454fa3c12ee None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:29.098377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9f486-0aac-4575-a0e1-a454fa3c12ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:29.150887 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f7094d6f-1d00-4c57-a77b-fe3494b69f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee012e49d4394539bd68d05305a6b7ea'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:29.152543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f7094d6f-1d00-4c57-a77b-fe3494b69f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee012e49d4394539bd68d05305a6b7ea'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:29.169066 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f7094d6f-1d00-4c57-a77b-fe3494b69f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee012e49d4394539bd68d05305a6b7ea'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:29.218223 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f7094d6f-1d00-4c57-a77b-fe3494b69f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee012e49d4394539bd68d05305a6b7ea'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:29.219016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-abb9f486-0aac-4575-a0e1-a454fa3c12ee None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:29.221280 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-abb9f486-0aac-4575-a0e1-a454fa3c12ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:29.243548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f7094d6f-1d00-4c57-a77b-fe3494b69f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee012e49d4394539bd68d05305a6b7ea'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:29.247264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:29.256720 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 913/1826] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:10:28 2022] DELETE /identity/v3/users/ee012e49d4394539bd68d05305a6b7ea => generated 0 bytes in 866 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:29.264937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0636f-2d7f-4752-bc68-4ef0c29c5a30 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:29.265406 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0636f-2d7f-4752-bc68-4ef0c29c5a30 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:29.265575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0636f-2d7f-4752-bc68-4ef0c29c5a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:29.330433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f8d0636f-2d7f-4752-bc68-4ef0c29c5a30 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:29.333057 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f8d0636f-2d7f-4752-bc68-4ef0c29c5a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:29.382831 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:29.580982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 914/1827] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:29.596072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 914/1828] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 513 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:29.709901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0636f-2d7f-4752-bc68-4ef0c29c5a30 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:29.712599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f8d0636f-2d7f-4752-bc68-4ef0c29c5a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:29.827499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9f486-0aac-4575-a0e1-a454fa3c12ee None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:29.829530 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-abb9f486-0aac-4575-a0e1-a454fa3c12ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:29.935386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0636f-2d7f-4752-bc68-4ef0c29c5a30 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:29.953093 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc0588-952b-4248-94a4-301d49e8c37f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:29.956034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc0588-952b-4248-94a4-301d49e8c37f service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:29.956034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc0588-952b-4248-94a4-301d49e8c37f service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:29.956034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc0588-952b-4248-94a4-301d49e8c37f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:30.009511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc0588-952b-4248-94a4-301d49e8c37f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:30.014021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc0588-952b-4248-94a4-301d49e8c37f service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:30.064220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 915/1829] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:30.071523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9f486-0aac-4575-a0e1-a454fa3c12ee None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:30.097433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967e12d-f943-45cb-a22e-8ea111a53b35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:30.098929 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967e12d-f943-45cb-a22e-8ea111a53b35 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:30.099564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967e12d-f943-45cb-a22e-8ea111a53b35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:30.100242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967e12d-f943-45cb-a22e-8ea111a53b35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:30.131955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967e12d-f943-45cb-a22e-8ea111a53b35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:30.142968 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967e12d-f943-45cb-a22e-8ea111a53b35 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:30.167440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 915/1830] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:33.443210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc0588-952b-4248-94a4-301d49e8c37f service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:33.448736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d5dc0588-952b-4248-94a4-301d49e8c37f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:33.495697 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc0588-952b-4248-94a4-301d49e8c37f service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:33.508620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9bfd6-780a-4c38-a465-4bfe3e2cf30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:33.510922 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9bfd6-780a-4c38-a465-4bfe3e2cf30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:33.511527 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9bfd6-780a-4c38-a465-4bfe3e2cf30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:33.512310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9bfd6-780a-4c38-a465-4bfe3e2cf30e admin admin] PATH_INFO: `/v3/projects/fb724d1f2a5d4154a1ca2f406a11653f` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:33.522318 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc9bfd6-780a-4c38-a465-4bfe3e2cf30e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb724d1f2a5d4154a1ca2f406a11653f)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:33.526321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc9bfd6-780a-4c38-a465-4bfe3e2cf30e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:33.741030 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4bc9bfd6-780a-4c38-a465-4bfe3e2cf30e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb724d1f2a5d4154a1ca2f406a11653f', 'request_id': 'req-4bc9bfd6-780a-4c38-a465-4bfe3e2cf30e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:33.766882 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4bc9bfd6-780a-4c38-a465-4bfe3e2cf30e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb724d1f2a5d4154a1ca2f406a11653f', 'request_id': 'req-4bc9bfd6-780a-4c38-a465-4bfe3e2cf30e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:33.781325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4bc9bfd6-780a-4c38-a465-4bfe3e2cf30e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb724d1f2a5d4154a1ca2f406a11653f', 'request_id': 'req-4bc9bfd6-780a-4c38-a465-4bfe3e2cf30e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:33.814711 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4bc9bfd6-780a-4c38-a465-4bfe3e2cf30e admin admin] The token cache is being invalidate because project fb724d1f2a5d4154a1ca2f406a11653f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:33.815296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4bc9bfd6-780a-4c38-a465-4bfe3e2cf30e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:33.816387 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4bc9bfd6-780a-4c38-a465-4bfe3e2cf30e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:33.818555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 916/1831] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:10:33 2022] DELETE /identity/v3/projects/fb724d1f2a5d4154a1ca2f406a11653f => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:33.824850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967e12d-f943-45cb-a22e-8ea111a53b35 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:33.837199 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967e12d-f943-45cb-a22e-8ea111a53b35 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:33.856336 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33167c3c-0a99-4ce2-9959-07d63a47c4c5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:33.858214 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33167c3c-0a99-4ce2-9959-07d63a47c4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:33.858510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33167c3c-0a99-4ce2-9959-07d63a47c4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:33.893203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-33167c3c-0a99-4ce2-9959-07d63a47c4c5 None None] MFA Rules not processed for user `6c62db4b38434cfe886108a72e4c42aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:33.899953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-33167c3c-0a99-4ce2-9959-07d63a47c4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:33.988130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:34.121255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba3d21-33d9-4979-996c-b0b7b6da9760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:34.138171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba3d21-33d9-4979-996c-b0b7b6da9760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:34.139036 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba3d21-33d9-4979-996c-b0b7b6da9760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:34.139715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba3d21-33d9-4979-996c-b0b7b6da9760 admin admin] PATH_INFO: `/v3/users/be775a6745ad46cd94aa223d824a0429` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:34.142051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba3d21-33d9-4979-996c-b0b7b6da9760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be775a6745ad46cd94aa223d824a0429)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:34.146293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba3d21-33d9-4979-996c-b0b7b6da9760 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:34.161809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 917/1832] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:34.161809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33167c3c-0a99-4ce2-9959-07d63a47c4c5 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:34.161809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-33167c3c-0a99-4ce2-9959-07d63a47c4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:34.176434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33167c3c-0a99-4ce2-9959-07d63a47c4c5 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:34.184553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:34.185496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:34.185779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:34.186056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46 admin admin] PATH_INFO: `/v3/projects/3f88472cf646401e83da00606a7daf12` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:34.187672 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f88472cf646401e83da00606a7daf12)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:34.189818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:34.286215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f88472cf646401e83da00606a7daf12', 'request_id': 'req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:34.298119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f88472cf646401e83da00606a7daf12', 'request_id': 'req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:34.307654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f4ba3d21-33d9-4979-996c-b0b7b6da9760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be775a6745ad46cd94aa223d824a0429'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:34.314813 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f88472cf646401e83da00606a7daf12', 'request_id': 'req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:34.323724 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f4ba3d21-33d9-4979-996c-b0b7b6da9760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be775a6745ad46cd94aa223d824a0429'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:34.332970 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46 admin admin] The token cache is being invalidate because project 3f88472cf646401e83da00606a7daf12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:34.333338 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:34.334033 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:34.336148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 918/1833] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:10:34 2022] DELETE /identity/v3/projects/3f88472cf646401e83da00606a7daf12 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:34.343957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:34.344809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f4ba3d21-33d9-4979-996c-b0b7b6da9760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be775a6745ad46cd94aa223d824a0429'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:34.348041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:34.362016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:34.420391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f4ba3d21-33d9-4979-996c-b0b7b6da9760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be775a6745ad46cd94aa223d824a0429'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:34.420820 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f4ba3d21-33d9-4979-996c-b0b7b6da9760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be775a6745ad46cd94aa223d824a0429'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:34.435741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 916/1834] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:10:33 2022] DELETE /identity/v3/users/be775a6745ad46cd94aa223d824a0429 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:34.818966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d13e2-0fbe-4f81-ac39-3ae2ec5f8c46 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:35.114891 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895840f-3708-496e-a66b-e8fa8a670e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:35.116538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895840f-3708-496e-a66b-e8fa8a670e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:35.117043 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895840f-3708-496e-a66b-e8fa8a670e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:35.117520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895840f-3708-496e-a66b-e8fa8a670e31 admin admin] PATH_INFO: `/v3/users/719adc8fb0bb4be0bfb677c5ce11dd8b` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:35.120652 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895840f-3708-496e-a66b-e8fa8a670e31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=719adc8fb0bb4be0bfb677c5ce11dd8b)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:35.126679 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895840f-3708-496e-a66b-e8fa8a670e31 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:35.130280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:35.279947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e895840f-3708-496e-a66b-e8fa8a670e31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '719adc8fb0bb4be0bfb677c5ce11dd8b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:35.281167 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e895840f-3708-496e-a66b-e8fa8a670e31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '719adc8fb0bb4be0bfb677c5ce11dd8b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:35.296557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e895840f-3708-496e-a66b-e8fa8a670e31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '719adc8fb0bb4be0bfb677c5ce11dd8b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:35.328190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e895840f-3708-496e-a66b-e8fa8a670e31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '719adc8fb0bb4be0bfb677c5ce11dd8b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:35.357592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e895840f-3708-496e-a66b-e8fa8a670e31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '719adc8fb0bb4be0bfb677c5ce11dd8b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:35.373374 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba3d21-33d9-4979-996c-b0b7b6da9760 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:35.375861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f4ba3d21-33d9-4979-996c-b0b7b6da9760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:35.410664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 919/1835] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:10:34 2022] DELETE /identity/v3/users/719adc8fb0bb4be0bfb677c5ce11dd8b => generated 0 bytes in 1068 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:10:35.422997 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba3d21-33d9-4979-996c-b0b7b6da9760 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:35.532575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd099bc5-65ac-405a-b3f8-ee5752c745ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:35.533916 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd099bc5-65ac-405a-b3f8-ee5752c745ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:35.534224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd099bc5-65ac-405a-b3f8-ee5752c745ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:35.534488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd099bc5-65ac-405a-b3f8-ee5752c745ff admin admin] PATH_INFO: `/v3/projects/b0a5e3e895e74c8f91f9a6176a431df0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:35.536130 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd099bc5-65ac-405a-b3f8-ee5752c745ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0a5e3e895e74c8f91f9a6176a431df0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:35.538250 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd099bc5-65ac-405a-b3f8-ee5752c745ff admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:35.618785 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd099bc5-65ac-405a-b3f8-ee5752c745ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0a5e3e895e74c8f91f9a6176a431df0', 'request_id': 'req-cd099bc5-65ac-405a-b3f8-ee5752c745ff'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:35.629955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd099bc5-65ac-405a-b3f8-ee5752c745ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0a5e3e895e74c8f91f9a6176a431df0', 'request_id': 'req-cd099bc5-65ac-405a-b3f8-ee5752c745ff'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:35.648802 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd099bc5-65ac-405a-b3f8-ee5752c745ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0a5e3e895e74c8f91f9a6176a431df0', 'request_id': 'req-cd099bc5-65ac-405a-b3f8-ee5752c745ff'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:35.678320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd099bc5-65ac-405a-b3f8-ee5752c745ff admin admin] The token cache is being invalidate because project b0a5e3e895e74c8f91f9a6176a431df0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:35.679290 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd099bc5-65ac-405a-b3f8-ee5752c745ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:35.680722 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd099bc5-65ac-405a-b3f8-ee5752c745ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:35.687565 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 917/1836] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:10:35 2022] DELETE /identity/v3/projects/b0a5e3e895e74c8f91f9a6176a431df0 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:10:35.689524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e95855-ade9-43b3-b81e-68192a00c8b2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:35.689949 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e95855-ade9-43b3-b81e-68192a00c8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:35.690397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e95855-ade9-43b3-b81e-68192a00c8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:35.767657 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e8e95855-ade9-43b3-b81e-68192a00c8b2 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:35.771894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e8e95855-ade9-43b3-b81e-68192a00c8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:36.082880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 920/1837] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:36.097672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcb554-b9f4-40f6-94da-2dda0c54cf5b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:36.097967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcb554-b9f4-40f6-94da-2dda0c54cf5b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:36.098286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcb554-b9f4-40f6-94da-2dda0c54cf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:36.136615 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-83bcb554-b9f4-40f6-94da-2dda0c54cf5b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:36.139948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-83bcb554-b9f4-40f6-94da-2dda0c54cf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:36.169484 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e95855-ade9-43b3-b81e-68192a00c8b2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:36.171606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e8e95855-ade9-43b3-b81e-68192a00c8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:36.181805 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 918/1838] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:10:36.188082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcb554-b9f4-40f6-94da-2dda0c54cf5b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:36.209299 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcb554-b9f4-40f6-94da-2dda0c54cf5b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:36.225992 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a3e1f-7a6d-486a-a516-cd2e33d4ec95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:36.227344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a3e1f-7a6d-486a-a516-cd2e33d4ec95 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:36.228012 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a3e1f-7a6d-486a-a516-cd2e33d4ec95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:36.229286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a3e1f-7a6d-486a-a516-cd2e33d4ec95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:36.231204 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a3e1f-7a6d-486a-a516-cd2e33d4ec95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:10:36.231884 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a3e1f-7a6d-486a-a516-cd2e33d4ec95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:36.232677 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e95855-ade9-43b3-b81e-68192a00c8b2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:36.236431 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a3e1f-7a6d-486a-a516-cd2e33d4ec95 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:36.278132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d52e18-054f-4cc3-b7ea-7c7ce8b1fa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:36.285643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d52e18-054f-4cc3-b7ea-7c7ce8b1fa61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:36.286125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d52e18-054f-4cc3-b7ea-7c7ce8b1fa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:36.286650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d52e18-054f-4cc3-b7ea-7c7ce8b1fa61 admin admin] PATH_INFO: `/v3/projects/609b19d2530043a59b76d9b99aebc5b7` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:36.293270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 919/1839] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:10:36.293767 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d52e18-054f-4cc3-b7ea-7c7ce8b1fa61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=609b19d2530043a59b76d9b99aebc5b7)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:36.294934 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a3e1f-7a6d-486a-a516-cd2e33d4ec95 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:36.297461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d52e18-054f-4cc3-b7ea-7c7ce8b1fa61 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:36.318849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a3e1f-7a6d-486a-a516-cd2e33d4ec95 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:36.328385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1eb3ca-2bef-42a1-8eb3-894b261f927a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:36.329684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1eb3ca-2bef-42a1-8eb3-894b261f927a admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:36.330173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1eb3ca-2bef-42a1-8eb3-894b261f927a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:36.330731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1eb3ca-2bef-42a1-8eb3-894b261f927a admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:36.332350 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1eb3ca-2bef-42a1-8eb3-894b261f927a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:36.338345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1eb3ca-2bef-42a1-8eb3-894b261f927a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:36.420377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 920/1840] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:36 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:36.424326 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1eb3ca-2bef-42a1-8eb3-894b261f927a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:36.508411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e5d52e18-054f-4cc3-b7ea-7c7ce8b1fa61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609b19d2530043a59b76d9b99aebc5b7', 'request_id': 'req-e5d52e18-054f-4cc3-b7ea-7c7ce8b1fa61'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:36.533007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e5d52e18-054f-4cc3-b7ea-7c7ce8b1fa61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609b19d2530043a59b76d9b99aebc5b7', 'request_id': 'req-e5d52e18-054f-4cc3-b7ea-7c7ce8b1fa61'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:36.557774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e5d52e18-054f-4cc3-b7ea-7c7ce8b1fa61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609b19d2530043a59b76d9b99aebc5b7', 'request_id': 'req-e5d52e18-054f-4cc3-b7ea-7c7ce8b1fa61'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:36.587221 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:36.594382 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e5d52e18-054f-4cc3-b7ea-7c7ce8b1fa61 admin admin] The token cache is being invalidate because project 609b19d2530043a59b76d9b99aebc5b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:36.594893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e5d52e18-054f-4cc3-b7ea-7c7ce8b1fa61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:36.600291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e5d52e18-054f-4cc3-b7ea-7c7ce8b1fa61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:36.608643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 921/1841] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:10:36 2022] DELETE /identity/v3/projects/609b19d2530043a59b76d9b99aebc5b7 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:36.611722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a8391-d29f-42c8-901d-74ca4a62dac6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:36.613869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a8391-d29f-42c8-901d-74ca4a62dac6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:36.614361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225a8391-d29f-42c8-901d-74ca4a62dac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:36.655652 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:36.660016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1eb3ca-2bef-42a1-8eb3-894b261f927a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:36.721197 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-225a8391-d29f-42c8-901d-74ca4a62dac6 None None] MFA Rules not processed for user `9e2a13c2cb864175a082de3ca8b166c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:36.724309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-225a8391-d29f-42c8-901d-74ca4a62dac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:36.881406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae306b12-3496-442e-8a47-749206e5cc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:36.887453 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae306b12-3496-442e-8a47-749206e5cc20 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:36.888138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae306b12-3496-442e-8a47-749206e5cc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:36.888618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae306b12-3496-442e-8a47-749206e5cc20 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:36.890881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae306b12-3496-442e-8a47-749206e5cc20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:36.908651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae306b12-3496-442e-8a47-749206e5cc20 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:36.964345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 921/1842] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:36 2022] POST /identity/v3/users => generated 291 bytes in 539 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:36.964345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae306b12-3496-442e-8a47-749206e5cc20 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:36.989298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ae306b12-3496-442e-8a47-749206e5cc20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:37.050434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae306b12-3496-442e-8a47-749206e5cc20 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:37.061831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 922/1843] 10.222.0.36 () {58 vars in 1015 bytes} [Thu Jul 21 09:10:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:37.063758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a8391-d29f-42c8-901d-74ca4a62dac6 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:37.069662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-225a8391-d29f-42c8-901d-74ca4a62dac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:37.069928 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152d9af-edff-4d22-82b8-ed17dbcf585c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:37.073910 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152d9af-edff-4d22-82b8-ed17dbcf585c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:37.074289 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152d9af-edff-4d22-82b8-ed17dbcf585c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:37.074582 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152d9af-edff-4d22-82b8-ed17dbcf585c admin admin] PATH_INFO: `/v3/users/ef1182541cd04cdd88d650d9b7846aed` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:37.083974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2152d9af-edff-4d22-82b8-ed17dbcf585c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef1182541cd04cdd88d650d9b7846aed)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:37.085902 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2152d9af-edff-4d22-82b8-ed17dbcf585c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:37.143086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225a8391-d29f-42c8-901d-74ca4a62dac6 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:37.196074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438b312-3260-4007-a1e0-84332cb64633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:37.196074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438b312-3260-4007-a1e0-84332cb64633 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:37.196074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438b312-3260-4007-a1e0-84332cb64633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:37.196074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438b312-3260-4007-a1e0-84332cb64633 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:37.197002 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438b312-3260-4007-a1e0-84332cb64633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:37.197002 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438b312-3260-4007-a1e0-84332cb64633 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:37.215959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 923/1844] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:37.215959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438b312-3260-4007-a1e0-84332cb64633 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:37.222070 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6438b312-3260-4007-a1e0-84332cb64633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:37.283417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438b312-3260-4007-a1e0-84332cb64633 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:37.308520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e8ef0-921e-4c79-919f-593fb996dba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:37.309675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e8ef0-921e-4c79-919f-593fb996dba3 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:37.311425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e8ef0-921e-4c79-919f-593fb996dba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:37.312235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e8ef0-921e-4c79-919f-593fb996dba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:37.371653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e8ef0-921e-4c79-919f-593fb996dba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:37.379189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e8ef0-921e-4c79-919f-593fb996dba3 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:37.415401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 924/1845] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:37.421163 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e8ef0-921e-4c79-919f-593fb996dba3 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:37.441713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2152d9af-edff-4d22-82b8-ed17dbcf585c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef1182541cd04cdd88d650d9b7846aed'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:37.450166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e8ef0-921e-4c79-919f-593fb996dba3 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:37.469071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2152d9af-edff-4d22-82b8-ed17dbcf585c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef1182541cd04cdd88d650d9b7846aed'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:37.472595 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bdebc-c112-497a-a4e0-7e1e17cb358d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:37.474344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bdebc-c112-497a-a4e0-7e1e17cb358d service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:37.475220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bdebc-c112-497a-a4e0-7e1e17cb358d service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:37.476356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bdebc-c112-497a-a4e0-7e1e17cb358d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:37.504062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bdebc-c112-497a-a4e0-7e1e17cb358d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:37.505685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bdebc-c112-497a-a4e0-7e1e17cb358d service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:37.528682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2152d9af-edff-4d22-82b8-ed17dbcf585c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef1182541cd04cdd88d650d9b7846aed'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:37.546101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 925/1846] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:37.546981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bdebc-c112-497a-a4e0-7e1e17cb358d service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:37.583656 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bdebc-c112-497a-a4e0-7e1e17cb358d service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:37.599263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4c352-bea5-45b3-af95-6c44a2091798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:37.599903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4c352-bea5-45b3-af95-6c44a2091798 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:37.600517 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4c352-bea5-45b3-af95-6c44a2091798 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:37.600784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4c352-bea5-45b3-af95-6c44a2091798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:37.642356 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2152d9af-edff-4d22-82b8-ed17dbcf585c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef1182541cd04cdd88d650d9b7846aed'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:37.643542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4c352-bea5-45b3-af95-6c44a2091798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:37.645072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4c352-bea5-45b3-af95-6c44a2091798 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:37.647948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2152d9af-edff-4d22-82b8-ed17dbcf585c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef1182541cd04cdd88d650d9b7846aed'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:37.691834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 926/1847] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:37.697210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4c352-bea5-45b3-af95-6c44a2091798 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:37.705990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-95d4c352-bea5-45b3-af95-6c44a2091798 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:37.723171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 922/1848] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:10:36 2022] DELETE /identity/v3/users/ef1182541cd04cdd88d650d9b7846aed => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:37.729328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152d9af-edff-4d22-82b8-ed17dbcf585c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:37.738699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2152d9af-edff-4d22-82b8-ed17dbcf585c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:37.770430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4c352-bea5-45b3-af95-6c44a2091798 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:37.917058 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:37.987991 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152d9af-edff-4d22-82b8-ed17dbcf585c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:38.007225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7ede5-7758-4e07-84b4-070750229eb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:38.008001 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7ede5-7758-4e07-84b4-070750229eb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:38.008296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7ede5-7758-4e07-84b4-070750229eb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:38.008578 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7ede5-7758-4e07-84b4-070750229eb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:38.018278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e2e51-5793-4255-9603-da952e156e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:38.019668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e2e51-5793-4255-9603-da952e156e7e admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:38.020347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e2e51-5793-4255-9603-da952e156e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:38.021075 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e2e51-5793-4255-9603-da952e156e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:38.023311 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e2e51-5793-4255-9603-da952e156e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:38.025114 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a1e7ede5-7758-4e07-84b4-070750229eb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:38.028940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e2e51-5793-4255-9603-da952e156e7e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:38.057991 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7ede5-7758-4e07-84b4-070750229eb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:38.083083 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 923/1849] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:38.085451 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e2e51-5793-4255-9603-da952e156e7e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:38.091672 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7ede5-7758-4e07-84b4-070750229eb5 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:38.115607 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e2e51-5793-4255-9603-da952e156e7e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:38.128267 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b75782-cc84-4966-84c9-2dbadc86c359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:38.130055 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b75782-cc84-4966-84c9-2dbadc86c359 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:38.131044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b75782-cc84-4966-84c9-2dbadc86c359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:38.131719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b75782-cc84-4966-84c9-2dbadc86c359 admin admin] PATH_INFO: `/v3/projects/0a8a353e5b9745a89f666622bf54510b/users/98fcee8404bb45d1a47501b86ae5eef7/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:38.133103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 927/1850] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:38.195724 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b75782-cc84-4966-84c9-2dbadc86c359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a8a353e5b9745a89f666622bf54510b, user_id=98fcee8404bb45d1a47501b86ae5eef7, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:38.200993 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b75782-cc84-4966-84c9-2dbadc86c359 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:38.264748 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 924/1851] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:38 2022] PUT /identity/v3/projects/0a8a353e5b9745a89f666622bf54510b/users/98fcee8404bb45d1a47501b86ae5eef7/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:38.272436 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08543716-a0a8-4389-9e58-42cac30de5b9 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:38.274058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08543716-a0a8-4389-9e58-42cac30de5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:38.276114 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08543716-a0a8-4389-9e58-42cac30de5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:38.418341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-08543716-a0a8-4389-9e58-42cac30de5b9 None None] MFA Rules not processed for user `98fcee8404bb45d1a47501b86ae5eef7`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:38.434892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-08543716-a0a8-4389-9e58-42cac30de5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:38.753286 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:38.796323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 928/1852] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 526 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:10:38.803189 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b75782-cc84-4966-84c9-2dbadc86c359 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:39.095877 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b75782-cc84-4966-84c9-2dbadc86c359 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:39.122080 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52073027-a825-4436-be72-a00eefb8ae97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:39.123182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52073027-a825-4436-be72-a00eefb8ae97 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:39.123998 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52073027-a825-4436-be72-a00eefb8ae97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:39.124668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52073027-a825-4436-be72-a00eefb8ae97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:39.125385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52073027-a825-4436-be72-a00eefb8ae97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:39.127573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52073027-a825-4436-be72-a00eefb8ae97 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:39.254588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 925/1853] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:38 2022] POST /identity/v3/projects => generated 379 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:39.255658 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08543716-a0a8-4389-9e58-42cac30de5b9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:39.281799 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08543716-a0a8-4389-9e58-42cac30de5b9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:39.459449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce0318-f754-4645-88c9-8b2eb7e5c2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:39.460618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce0318-f754-4645-88c9-8b2eb7e5c2d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:39.461042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce0318-f754-4645-88c9-8b2eb7e5c2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:39.461791 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce0318-f754-4645-88c9-8b2eb7e5c2d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:39.462315 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce0318-f754-4645-88c9-8b2eb7e5c2d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:39.465444 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce0318-f754-4645-88c9-8b2eb7e5c2d7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:39.543939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 929/1854] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:39 2022] POST /identity/v3/users => generated 296 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:39.549336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52073027-a825-4436-be72-a00eefb8ae97 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:39.566040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52073027-a825-4436-be72-a00eefb8ae97 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:39.600374 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ebcaf-99d1-4911-9f72-e61e59bb1b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:39.606941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ebcaf-99d1-4911-9f72-e61e59bb1b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:39.608117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ebcaf-99d1-4911-9f72-e61e59bb1b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:39.610523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ebcaf-99d1-4911-9f72-e61e59bb1b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:39.613254 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ebcaf-99d1-4911-9f72-e61e59bb1b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:39.617804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ebcaf-99d1-4911-9f72-e61e59bb1b36 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:39.646078 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 926/1855] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:39.651488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce0318-f754-4645-88c9-8b2eb7e5c2d7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:39.664409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce0318-f754-4645-88c9-8b2eb7e5c2d7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:39.670305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d993c-0ee2-45b8-be05-aedfbe993078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:39.671244 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d993c-0ee2-45b8-be05-aedfbe993078 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:39.671532 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d993c-0ee2-45b8-be05-aedfbe993078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:39.672266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d993c-0ee2-45b8-be05-aedfbe993078 admin admin] PATH_INFO: `/v3/projects/f3b7e5226e474f26a7adebf205d83c10/users/5c2c6deb2bfd4574bfa7979dcca49e75/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:39.712606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d993c-0ee2-45b8-be05-aedfbe993078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3b7e5226e474f26a7adebf205d83c10, user_id=5c2c6deb2bfd4574bfa7979dcca49e75, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:39.717192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d993c-0ee2-45b8-be05-aedfbe993078 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:39.753632 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ebcaf-99d1-4911-9f72-e61e59bb1b36 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:39.758662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 930/1856] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:39 2022] PUT /identity/v3/projects/f3b7e5226e474f26a7adebf205d83c10/users/5c2c6deb2bfd4574bfa7979dcca49e75/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:39.797573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ebcaf-99d1-4911-9f72-e61e59bb1b36 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:40.009981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b269f7f-47b5-4d8e-82ac-c22872e765c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:40.010885 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b269f7f-47b5-4d8e-82ac-c22872e765c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:40.011246 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b269f7f-47b5-4d8e-82ac-c22872e765c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:40.011595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b269f7f-47b5-4d8e-82ac-c22872e765c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:40.014116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b269f7f-47b5-4d8e-82ac-c22872e765c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:40.016994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b269f7f-47b5-4d8e-82ac-c22872e765c5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:40.037300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d993c-0ee2-45b8-be05-aedfbe993078 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:40.049648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 927/1857] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:40.056150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d993c-0ee2-45b8-be05-aedfbe993078 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:40.074854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ece06-05d8-4042-a5ff-6eea1ae22272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:40.078405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ece06-05d8-4042-a5ff-6eea1ae22272 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:40.081545 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ece06-05d8-4042-a5ff-6eea1ae22272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:40.082202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ece06-05d8-4042-a5ff-6eea1ae22272 admin admin] PATH_INFO: `/v3/projects/f3b7e5226e474f26a7adebf205d83c10/users/5c2c6deb2bfd4574bfa7979dcca49e75/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:40.091876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ece06-05d8-4042-a5ff-6eea1ae22272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3b7e5226e474f26a7adebf205d83c10, user_id=5c2c6deb2bfd4574bfa7979dcca49e75, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:40.102511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ece06-05d8-4042-a5ff-6eea1ae22272 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:40.149890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 931/1858] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:40 2022] PUT /identity/v3/projects/f3b7e5226e474f26a7adebf205d83c10/users/5c2c6deb2bfd4574bfa7979dcca49e75/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:40.155455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1ced5-0928-4838-867d-13afbc4f58a4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:40.156164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1ced5-0928-4838-867d-13afbc4f58a4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:40.156645 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1ced5-0928-4838-867d-13afbc4f58a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:40.231604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4ec1ced5-0928-4838-867d-13afbc4f58a4 None None] MFA Rules not processed for user `5c2c6deb2bfd4574bfa7979dcca49e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:40.237518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4ec1ced5-0928-4838-867d-13afbc4f58a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:40.282747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:40.508749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 928/1859] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:40.518649 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ece06-05d8-4042-a5ff-6eea1ae22272 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:40.520636 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a07ece06-05d8-4042-a5ff-6eea1ae22272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:40.683971 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ece06-05d8-4042-a5ff-6eea1ae22272 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:40.693426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac359b5f-63d2-4f78-b782-74b5f8ea3d5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:40.698243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac359b5f-63d2-4f78-b782-74b5f8ea3d5a service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:40.698562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac359b5f-63d2-4f78-b782-74b5f8ea3d5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:40.698841 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac359b5f-63d2-4f78-b782-74b5f8ea3d5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:40.738570 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac359b5f-63d2-4f78-b782-74b5f8ea3d5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:40.740456 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac359b5f-63d2-4f78-b782-74b5f8ea3d5a service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:40.786613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 932/1860] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:10:41.104734 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1ced5-0928-4838-867d-13afbc4f58a4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:41.134934 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1ced5-0928-4838-867d-13afbc4f58a4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:41.150019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af04024-0222-4fee-9c4b-188b3012ac9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:41.151260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af04024-0222-4fee-9c4b-188b3012ac9d service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:41.153756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af04024-0222-4fee-9c4b-188b3012ac9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:41.153756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af04024-0222-4fee-9c4b-188b3012ac9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:41.306739 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af04024-0222-4fee-9c4b-188b3012ac9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:41.309376 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af04024-0222-4fee-9c4b-188b3012ac9d service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:41.340098 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 929/1861] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:41.426004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac359b5f-63d2-4f78-b782-74b5f8ea3d5a service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:41.428563 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ac359b5f-63d2-4f78-b782-74b5f8ea3d5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:41.594160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:41.668268 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac359b5f-63d2-4f78-b782-74b5f8ea3d5a service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:41.674258 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d639d-ed1e-4745-a97c-319e30d743fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:41.675437 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d639d-ed1e-4745-a97c-319e30d743fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:41.676399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d639d-ed1e-4745-a97c-319e30d743fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:41.677026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d639d-ed1e-4745-a97c-319e30d743fd admin admin] PATH_INFO: `/v3/projects/564af1dfb6f9439aab14191a374130b0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:41.679662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d639d-ed1e-4745-a97c-319e30d743fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=564af1dfb6f9439aab14191a374130b0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:41.683511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d639d-ed1e-4745-a97c-319e30d743fd admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:41.811678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-973d639d-ed1e-4745-a97c-319e30d743fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '564af1dfb6f9439aab14191a374130b0', 'request_id': 'req-973d639d-ed1e-4745-a97c-319e30d743fd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:41.824386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-973d639d-ed1e-4745-a97c-319e30d743fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '564af1dfb6f9439aab14191a374130b0', 'request_id': 'req-973d639d-ed1e-4745-a97c-319e30d743fd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:41.844895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-973d639d-ed1e-4745-a97c-319e30d743fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '564af1dfb6f9439aab14191a374130b0', 'request_id': 'req-973d639d-ed1e-4745-a97c-319e30d743fd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:41.911246 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-973d639d-ed1e-4745-a97c-319e30d743fd admin admin] The token cache is being invalidate because project 564af1dfb6f9439aab14191a374130b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:41.912355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-973d639d-ed1e-4745-a97c-319e30d743fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:41.921258 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-973d639d-ed1e-4745-a97c-319e30d743fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:41.932072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5020f71-1fe6-4128-9743-a6fcf3643146 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:41.932421 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5020f71-1fe6-4128-9743-a6fcf3643146 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:41.932633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5020f71-1fe6-4128-9743-a6fcf3643146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:41.948524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 933/1862] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:10:41 2022] DELETE /identity/v3/projects/564af1dfb6f9439aab14191a374130b0 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:42.001988 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d5020f71-1fe6-4128-9743-a6fcf3643146 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:42.003724 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d5020f71-1fe6-4128-9743-a6fcf3643146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:42.265442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 930/1863] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:42.274033 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c4128-447e-402b-b194-88469507aa88 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:42.274333 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c4128-447e-402b-b194-88469507aa88 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:42.274594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c4128-447e-402b-b194-88469507aa88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:42.342293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ba9c4128-447e-402b-b194-88469507aa88 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:42.343992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ba9c4128-447e-402b-b194-88469507aa88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:42.351401 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5020f71-1fe6-4128-9743-a6fcf3643146 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:42.357539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d5020f71-1fe6-4128-9743-a6fcf3643146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:42.403541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 934/1864] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:42.413526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c4128-447e-402b-b194-88469507aa88 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:42.415018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5020f71-1fe6-4128-9743-a6fcf3643146 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:42.444386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32df418-f73d-4262-87d1-f3537e393b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:42.450028 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df418-f73d-4262-87d1-f3537e393b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:42.450330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df418-f73d-4262-87d1-f3537e393b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:42.452682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df418-f73d-4262-87d1-f3537e393b78 admin admin] PATH_INFO: `/v3/users/98fcee8404bb45d1a47501b86ae5eef7` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:42.456123 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32df418-f73d-4262-87d1-f3537e393b78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98fcee8404bb45d1a47501b86ae5eef7)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:42.458388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32df418-f73d-4262-87d1-f3537e393b78 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:42.459454 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c4128-447e-402b-b194-88469507aa88 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:42.471010 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828fd42f-3b85-435b-b8e7-b748ae116bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:42.472045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fd42f-3b85-435b-b8e7-b748ae116bff admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:42.472737 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fd42f-3b85-435b-b8e7-b748ae116bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:42.489218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fd42f-3b85-435b-b8e7-b748ae116bff admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:42.490586 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828fd42f-3b85-435b-b8e7-b748ae116bff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:10:42.494575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828fd42f-3b85-435b-b8e7-b748ae116bff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:42.502899 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828fd42f-3b85-435b-b8e7-b748ae116bff admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:42.562858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 935/1865] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:10:42.570455 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828fd42f-3b85-435b-b8e7-b748ae116bff admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:42.582165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-828fd42f-3b85-435b-b8e7-b748ae116bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:42.633720 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828fd42f-3b85-435b-b8e7-b748ae116bff admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:42.655097 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34310f-875c-4480-9d3e-e1ff6beca16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:42.656074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34310f-875c-4480-9d3e-e1ff6beca16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:42.656375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34310f-875c-4480-9d3e-e1ff6beca16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:42.656736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34310f-875c-4480-9d3e-e1ff6beca16e admin admin] PATH_INFO: `/v3/users/fdb5af6be42848b1aca7be261ae5ae40` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:42.661473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34310f-875c-4480-9d3e-e1ff6beca16e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdb5af6be42848b1aca7be261ae5ae40)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:42.663619 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34310f-875c-4480-9d3e-e1ff6beca16e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:42.665507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d32df418-f73d-4262-87d1-f3537e393b78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98fcee8404bb45d1a47501b86ae5eef7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:42.672447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:42.688598 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d32df418-f73d-4262-87d1-f3537e393b78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98fcee8404bb45d1a47501b86ae5eef7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:42.715096 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d32df418-f73d-4262-87d1-f3537e393b78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98fcee8404bb45d1a47501b86ae5eef7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:42.743687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d32df418-f73d-4262-87d1-f3537e393b78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98fcee8404bb45d1a47501b86ae5eef7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:42.745588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d32df418-f73d-4262-87d1-f3537e393b78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98fcee8404bb45d1a47501b86ae5eef7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:42.767844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 931/1866] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:10:42 2022] DELETE /identity/v3/users/98fcee8404bb45d1a47501b86ae5eef7 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:10:42.768930 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32df418-f73d-4262-87d1-f3537e393b78 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:42.772496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d32df418-f73d-4262-87d1-f3537e393b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:42.831134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-eb34310f-875c-4480-9d3e-e1ff6beca16e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdb5af6be42848b1aca7be261ae5ae40'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:42.832521 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-eb34310f-875c-4480-9d3e-e1ff6beca16e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdb5af6be42848b1aca7be261ae5ae40'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:42.886719 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-eb34310f-875c-4480-9d3e-e1ff6beca16e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdb5af6be42848b1aca7be261ae5ae40'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:42.959537 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-eb34310f-875c-4480-9d3e-e1ff6beca16e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdb5af6be42848b1aca7be261ae5ae40'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:42.983984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32df418-f73d-4262-87d1-f3537e393b78 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:42.993057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ff7fe-fa27-4197-9b9f-f9aef9604abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:42.994031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ff7fe-fa27-4197-9b9f-f9aef9604abb admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:42.998059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ff7fe-fa27-4197-9b9f-f9aef9604abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:42.998554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ff7fe-fa27-4197-9b9f-f9aef9604abb admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:42.999678 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ff7fe-fa27-4197-9b9f-f9aef9604abb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:43.007002 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-eb34310f-875c-4480-9d3e-e1ff6beca16e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdb5af6be42848b1aca7be261ae5ae40'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:43.007852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ff7fe-fa27-4197-9b9f-f9aef9604abb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:43.058117 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 936/1867] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:10:42 2022] DELETE /identity/v3/users/fdb5af6be42848b1aca7be261ae5ae40 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:43.070252 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7febd-dafb-4a90-ad90-6e91eadd8be5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:43.071398 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7febd-dafb-4a90-ad90-6e91eadd8be5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:43.074734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7febd-dafb-4a90-ad90-6e91eadd8be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:43.084771 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 932/1868] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:42 2022] POST /identity/v3/projects => generated 369 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:43.091205 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438424dd-e8ec-456b-9070-2a56a34f3a71 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:43.093319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438424dd-e8ec-456b-9070-2a56a34f3a71 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:43.093801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438424dd-e8ec-456b-9070-2a56a34f3a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:43.171103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-438424dd-e8ec-456b-9070-2a56a34f3a71 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:43.172156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-65e7febd-dafb-4a90-ad90-6e91eadd8be5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:43.174558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-65e7febd-dafb-4a90-ad90-6e91eadd8be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:43.178465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-438424dd-e8ec-456b-9070-2a56a34f3a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:43.354715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:43.427050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 933/1869] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:43.428815 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438424dd-e8ec-456b-9070-2a56a34f3a71 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:43.436642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-438424dd-e8ec-456b-9070-2a56a34f3a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:43.466576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438424dd-e8ec-456b-9070-2a56a34f3a71 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:43.484658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe5b07-50ac-405e-80d6-16187f1a3a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:43.485666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe5b07-50ac-405e-80d6-16187f1a3a87 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:43.486276 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe5b07-50ac-405e-80d6-16187f1a3a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:43.486894 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe5b07-50ac-405e-80d6-16187f1a3a87 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:43.487822 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe5b07-50ac-405e-80d6-16187f1a3a87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:43.490328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fe5b07-50ac-405e-80d6-16187f1a3a87 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:43.596470 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 934/1870] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:43 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:43.609970 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe5b07-50ac-405e-80d6-16187f1a3a87 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:43.667800 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 937/1871] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 608 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:43.688820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7febd-dafb-4a90-ad90-6e91eadd8be5 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:43.691133 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-65e7febd-dafb-4a90-ad90-6e91eadd8be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:43.715739 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe5b07-50ac-405e-80d6-16187f1a3a87 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:43.768019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ec5e7-69a3-4cd3-84f7-ae37d149f9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:43.770908 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ec5e7-69a3-4cd3-84f7-ae37d149f9ac admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:43.774562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ec5e7-69a3-4cd3-84f7-ae37d149f9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:43.775015 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ec5e7-69a3-4cd3-84f7-ae37d149f9ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:43.781236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ec5e7-69a3-4cd3-84f7-ae37d149f9ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:43.805313 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ec5e7-69a3-4cd3-84f7-ae37d149f9ac admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:43.888931 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 935/1872] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:10:43.888931 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe8b60-a248-466f-b45e-3cabe8e5a3aa None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:43.891099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe8b60-a248-466f-b45e-3cabe8e5a3aa None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:43.891099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe8b60-a248-466f-b45e-3cabe8e5a3aa None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:43.893397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 936/1873] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jul 21 09:10:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:10:43.894794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe8b60-a248-466f-b45e-3cabe8e5a3aa None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:43.899824 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-febe8b60-a248-466f-b45e-3cabe8e5a3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:44.003223 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7febd-dafb-4a90-ad90-6e91eadd8be5 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:44.023207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075ea60-8522-4e4d-9196-a85388c70ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:44.024261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075ea60-8522-4e4d-9196-a85388c70ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:44.024757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075ea60-8522-4e4d-9196-a85388c70ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:44.025900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075ea60-8522-4e4d-9196-a85388c70ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:44.039212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:44.091786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075ea60-8522-4e4d-9196-a85388c70ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:44.094623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075ea60-8522-4e4d-9196-a85388c70ea6 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:44.165152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 938/1874] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:10:44.168655 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075ea60-8522-4e4d-9196-a85388c70ea6 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:44.201877 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe8b60-a248-466f-b45e-3cabe8e5a3aa None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:44.207342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480edb3-c31b-4865-b647-7711c377cabf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:44.208129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480edb3-c31b-4865-b647-7711c377cabf service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:44.208496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480edb3-c31b-4865-b647-7711c377cabf service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:44.208772 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480edb3-c31b-4865-b647-7711c377cabf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:44.227890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075ea60-8522-4e4d-9196-a85388c70ea6 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:44.238131 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480edb3-c31b-4865-b647-7711c377cabf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:44.240053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480edb3-c31b-4865-b647-7711c377cabf service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:44.243034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359536f5-a0ea-4621-a678-2b26ef312726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:44.246780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359536f5-a0ea-4621-a678-2b26ef312726 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:44.247523 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359536f5-a0ea-4621-a678-2b26ef312726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:44.248172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359536f5-a0ea-4621-a678-2b26ef312726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:44.249758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359536f5-a0ea-4621-a678-2b26ef312726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:44.254964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359536f5-a0ea-4621-a678-2b26ef312726 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:44.285668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 937/1875] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:44.287204 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eec60a-0210-4300-b08a-1beb0dc1510f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:44.287494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eec60a-0210-4300-b08a-1beb0dc1510f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:44.287776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eec60a-0210-4300-b08a-1beb0dc1510f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:44.294028 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 939/1876] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:44.295236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359536f5-a0ea-4621-a678-2b26ef312726 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:44.320279 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359536f5-a0ea-4621-a678-2b26ef312726 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:44.328115 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc38053-ccaf-4427-9d64-e54ad4c2f5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:44.329480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc38053-ccaf-4427-9d64-e54ad4c2f5b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:44.329882 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc38053-ccaf-4427-9d64-e54ad4c2f5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:44.330269 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc38053-ccaf-4427-9d64-e54ad4c2f5b5 admin admin] PATH_INFO: `/v3/projects/9a23621f15104d378cb86f0b3914082e/users/345df51fe411448fbd4b183233e899fd/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:44.364156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-82eec60a-0210-4300-b08a-1beb0dc1510f None None] MFA Rules not processed for user `c48d41dcf9b344a8b2fec77f4dd75fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:44.372187 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-82eec60a-0210-4300-b08a-1beb0dc1510f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:44.398627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc38053-ccaf-4427-9d64-e54ad4c2f5b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a23621f15104d378cb86f0b3914082e, user_id=345df51fe411448fbd4b183233e899fd, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:44.403779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc38053-ccaf-4427-9d64-e54ad4c2f5b5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:44.431347 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 940/1877] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:44 2022] PUT /identity/v3/projects/9a23621f15104d378cb86f0b3914082e/users/345df51fe411448fbd4b183233e899fd/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:44.439945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb7c7a-25bc-4342-9afd-ffc2a95d21d5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:44.440223 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb7c7a-25bc-4342-9afd-ffc2a95d21d5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:44.441088 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb7c7a-25bc-4342-9afd-ffc2a95d21d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:44.485874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-eacb7c7a-25bc-4342-9afd-ffc2a95d21d5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:44.488563 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-eacb7c7a-25bc-4342-9afd-ffc2a95d21d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:44.567963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 938/1878] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jul 21 09:10:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:44.681892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 941/1879] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:44.916652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eec60a-0210-4300-b08a-1beb0dc1510f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:44.957015 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eec60a-0210-4300-b08a-1beb0dc1510f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:45.173254 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5f432-01f5-4415-afe6-eb506864cb73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:45.176613 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5f432-01f5-4415-afe6-eb506864cb73 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:45.176613 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5f432-01f5-4415-afe6-eb506864cb73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:45.176613 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5f432-01f5-4415-afe6-eb506864cb73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:45.224403 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5f432-01f5-4415-afe6-eb506864cb73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:45.227591 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5f432-01f5-4415-afe6-eb506864cb73 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:45.293655 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 939/1880] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:45.958242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb7c7a-25bc-4342-9afd-ffc2a95d21d5 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:45.962293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-eacb7c7a-25bc-4342-9afd-ffc2a95d21d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:46.061632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:46.152170 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb7c7a-25bc-4342-9afd-ffc2a95d21d5 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:46.172770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74e168-54f0-403e-8ecb-87ed2a61c580 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:46.174380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74e168-54f0-403e-8ecb-87ed2a61c580 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:46.174380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74e168-54f0-403e-8ecb-87ed2a61c580 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:46.176659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74e168-54f0-403e-8ecb-87ed2a61c580 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:46.224847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74e168-54f0-403e-8ecb-87ed2a61c580 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:46.233026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74e168-54f0-403e-8ecb-87ed2a61c580 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:46.316285 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10163f67-7e93-41a0-b2ce-b48863bf7dd4 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:46.316905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10163f67-7e93-41a0-b2ce-b48863bf7dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:46.317441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10163f67-7e93-41a0-b2ce-b48863bf7dd4 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:46.320767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 940/1881] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jul 21 09:10:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:10:46.364405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 942/1882] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:46.376986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb6677-fa48-4d99-b3c0-c65cc3fcd119 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:46.378307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb6677-fa48-4d99-b3c0-c65cc3fcd119 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:46.379146 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb6677-fa48-4d99-b3c0-c65cc3fcd119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:46.476144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1aeb6677-fa48-4d99-b3c0-c65cc3fcd119 None None] MFA Rules not processed for user `c48d41dcf9b344a8b2fec77f4dd75fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:46.479403 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1aeb6677-fa48-4d99-b3c0-c65cc3fcd119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:46.491693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 941/1883] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jul 21 09:10:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:46.524830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74e168-54f0-403e-8ecb-87ed2a61c580 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:46.526985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3c74e168-54f0-403e-8ecb-87ed2a61c580 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:46.545675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74e168-54f0-403e-8ecb-87ed2a61c580 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:46.551944 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c054fff-0e62-4acd-9287-a533f45b0058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:46.553358 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c054fff-0e62-4acd-9287-a533f45b0058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:46.553905 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c054fff-0e62-4acd-9287-a533f45b0058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:46.554403 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c054fff-0e62-4acd-9287-a533f45b0058 admin admin] PATH_INFO: `/v3/projects/0a8a353e5b9745a89f666622bf54510b` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:46.557172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c054fff-0e62-4acd-9287-a533f45b0058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a8a353e5b9745a89f666622bf54510b)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:46.559776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c054fff-0e62-4acd-9287-a533f45b0058 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:46.715722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7c054fff-0e62-4acd-9287-a533f45b0058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a8a353e5b9745a89f666622bf54510b', 'request_id': 'req-7c054fff-0e62-4acd-9287-a533f45b0058'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:46.734516 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7c054fff-0e62-4acd-9287-a533f45b0058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a8a353e5b9745a89f666622bf54510b', 'request_id': 'req-7c054fff-0e62-4acd-9287-a533f45b0058'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:46.759153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7c054fff-0e62-4acd-9287-a533f45b0058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a8a353e5b9745a89f666622bf54510b', 'request_id': 'req-7c054fff-0e62-4acd-9287-a533f45b0058'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:46.826821 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7c054fff-0e62-4acd-9287-a533f45b0058 admin admin] The token cache is being invalidate because project 0a8a353e5b9745a89f666622bf54510b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:46.827492 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7c054fff-0e62-4acd-9287-a533f45b0058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:46.829001 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7c054fff-0e62-4acd-9287-a533f45b0058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:46.833166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 943/1884] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:10:46 2022] DELETE /identity/v3/projects/0a8a353e5b9745a89f666622bf54510b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:10:46.839558 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb6677-fa48-4d99-b3c0-c65cc3fcd119 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:46.862953 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb6677-fa48-4d99-b3c0-c65cc3fcd119 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:46.955831 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:46.971265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866c28f-000d-4bca-984f-e0d536fa55bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:46.973193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866c28f-000d-4bca-984f-e0d536fa55bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:46.973661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866c28f-000d-4bca-984f-e0d536fa55bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:46.974125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866c28f-000d-4bca-984f-e0d536fa55bb admin admin] PATH_INFO: `/v3/users/5c2c6deb2bfd4574bfa7979dcca49e75` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:46.976581 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c866c28f-000d-4bca-984f-e0d536fa55bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c2c6deb2bfd4574bfa7979dcca49e75)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:46.981855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c866c28f-000d-4bca-984f-e0d536fa55bb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:47.144971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c866c28f-000d-4bca-984f-e0d536fa55bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c2c6deb2bfd4574bfa7979dcca49e75'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:47.170531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c866c28f-000d-4bca-984f-e0d536fa55bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c2c6deb2bfd4574bfa7979dcca49e75'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:47.181078 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c866c28f-000d-4bca-984f-e0d536fa55bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c2c6deb2bfd4574bfa7979dcca49e75'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:47.197876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c866c28f-000d-4bca-984f-e0d536fa55bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c2c6deb2bfd4574bfa7979dcca49e75'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:47.198736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c866c28f-000d-4bca-984f-e0d536fa55bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c2c6deb2bfd4574bfa7979dcca49e75'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:47.209355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 942/1885] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:10:46 2022] DELETE /identity/v3/users/5c2c6deb2bfd4574bfa7979dcca49e75 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:47.290308 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c054fff-0e62-4acd-9287-a533f45b0058 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:47.294361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7c054fff-0e62-4acd-9287-a533f45b0058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:47.574881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c054fff-0e62-4acd-9287-a533f45b0058 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:47.641656 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866c28f-000d-4bca-984f-e0d536fa55bb admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:47.643321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c866c28f-000d-4bca-984f-e0d536fa55bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:47.647939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:47.677670 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b155c9a-08f6-4e6a-bacc-e6dc03054bc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:47.678380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b155c9a-08f6-4e6a-bacc-e6dc03054bc4 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:47.678764 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b155c9a-08f6-4e6a-bacc-e6dc03054bc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:47.679041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b155c9a-08f6-4e6a-bacc-e6dc03054bc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:47.681110 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5b155c9a-08f6-4e6a-bacc-e6dc03054bc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:47.695621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b155c9a-08f6-4e6a-bacc-e6dc03054bc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:47.698139 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b155c9a-08f6-4e6a-bacc-e6dc03054bc4 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:47.716017 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 944/1886] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:47.802747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866c28f-000d-4bca-984f-e0d536fa55bb admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:47.831451 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b56a1-5193-4e8c-97eb-520597b16853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:47.832755 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b56a1-5193-4e8c-97eb-520597b16853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:47.833200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b56a1-5193-4e8c-97eb-520597b16853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:47.833611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b56a1-5193-4e8c-97eb-520597b16853 admin admin] PATH_INFO: `/v3/projects/d16136c7ecd548cdbd1b5d123f21e68f` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:47.836290 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b56a1-5193-4e8c-97eb-520597b16853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d16136c7ecd548cdbd1b5d123f21e68f)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:47.842941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b56a1-5193-4e8c-97eb-520597b16853 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:47.945319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-585b56a1-5193-4e8c-97eb-520597b16853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd16136c7ecd548cdbd1b5d123f21e68f', 'request_id': 'req-585b56a1-5193-4e8c-97eb-520597b16853'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:47.975276 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-585b56a1-5193-4e8c-97eb-520597b16853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd16136c7ecd548cdbd1b5d123f21e68f', 'request_id': 'req-585b56a1-5193-4e8c-97eb-520597b16853'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:47.997206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-585b56a1-5193-4e8c-97eb-520597b16853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd16136c7ecd548cdbd1b5d123f21e68f', 'request_id': 'req-585b56a1-5193-4e8c-97eb-520597b16853'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:48.037534 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-585b56a1-5193-4e8c-97eb-520597b16853 admin admin] The token cache is being invalidate because project d16136c7ecd548cdbd1b5d123f21e68f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:48.038205 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-585b56a1-5193-4e8c-97eb-520597b16853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:48.039593 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-585b56a1-5193-4e8c-97eb-520597b16853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:48.048955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 943/1887] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:10:47 2022] DELETE /identity/v3/projects/d16136c7ecd548cdbd1b5d123f21e68f => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:10:48.050252 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b155c9a-08f6-4e6a-bacc-e6dc03054bc4 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:48.078315 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b155c9a-08f6-4e6a-bacc-e6dc03054bc4 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:48.285812 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5820b6f-d88e-44da-b309-0ac70fcd2c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:48.287524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5820b6f-d88e-44da-b309-0ac70fcd2c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:48.287524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5820b6f-d88e-44da-b309-0ac70fcd2c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:48.287866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5820b6f-d88e-44da-b309-0ac70fcd2c56 admin admin] PATH_INFO: `/v3/users/53df719c21dc4049a1292f33dfdcef78` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:48.289382 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5820b6f-d88e-44da-b309-0ac70fcd2c56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53df719c21dc4049a1292f33dfdcef78)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:48.292075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5820b6f-d88e-44da-b309-0ac70fcd2c56 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:48.470549 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c5820b6f-d88e-44da-b309-0ac70fcd2c56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53df719c21dc4049a1292f33dfdcef78'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:48.475470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c5820b6f-d88e-44da-b309-0ac70fcd2c56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53df719c21dc4049a1292f33dfdcef78'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:48.499704 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c5820b6f-d88e-44da-b309-0ac70fcd2c56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53df719c21dc4049a1292f33dfdcef78'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:48.545442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c5820b6f-d88e-44da-b309-0ac70fcd2c56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53df719c21dc4049a1292f33dfdcef78'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:48.561850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c5820b6f-d88e-44da-b309-0ac70fcd2c56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53df719c21dc4049a1292f33dfdcef78'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:48.581075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 945/1888] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:10:48 2022] DELETE /identity/v3/users/53df719c21dc4049a1292f33dfdcef78 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:48.583001 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b56a1-5193-4e8c-97eb-520597b16853 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:48.585703 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-585b56a1-5193-4e8c-97eb-520597b16853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:48.600816 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:48.620489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b56a1-5193-4e8c-97eb-520597b16853 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:48.821620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad98ae3-0a08-4218-97ae-06c4e3433f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:48.827811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad98ae3-0a08-4218-97ae-06c4e3433f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:48.828584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad98ae3-0a08-4218-97ae-06c4e3433f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:48.830123 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad98ae3-0a08-4218-97ae-06c4e3433f5d admin admin] PATH_INFO: `/v3/projects/f3b7e5226e474f26a7adebf205d83c10` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:48.840423 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad98ae3-0a08-4218-97ae-06c4e3433f5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3b7e5226e474f26a7adebf205d83c10)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:48.854134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad98ae3-0a08-4218-97ae-06c4e3433f5d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:49.066632 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8ad98ae3-0a08-4218-97ae-06c4e3433f5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3b7e5226e474f26a7adebf205d83c10', 'request_id': 'req-8ad98ae3-0a08-4218-97ae-06c4e3433f5d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:49.087654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8ad98ae3-0a08-4218-97ae-06c4e3433f5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3b7e5226e474f26a7adebf205d83c10', 'request_id': 'req-8ad98ae3-0a08-4218-97ae-06c4e3433f5d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:49.105951 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8ad98ae3-0a08-4218-97ae-06c4e3433f5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3b7e5226e474f26a7adebf205d83c10', 'request_id': 'req-8ad98ae3-0a08-4218-97ae-06c4e3433f5d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:49.132661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8ad98ae3-0a08-4218-97ae-06c4e3433f5d admin admin] The token cache is being invalidate because project f3b7e5226e474f26a7adebf205d83c10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:49.133168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8ad98ae3-0a08-4218-97ae-06c4e3433f5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:49.135413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8ad98ae3-0a08-4218-97ae-06c4e3433f5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:49.138993 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 944/1889] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:10:48 2022] DELETE /identity/v3/projects/f3b7e5226e474f26a7adebf205d83c10 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:49.144987 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66794425-c40f-4650-b8c6-ac0b2ef22ff6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:49.145301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66794425-c40f-4650-b8c6-ac0b2ef22ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:49.145566 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66794425-c40f-4650-b8c6-ac0b2ef22ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:49.168516 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-66794425-c40f-4650-b8c6-ac0b2ef22ff6 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:49.170880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-66794425-c40f-4650-b8c6-ac0b2ef22ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:49.249868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:49.291833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 946/1890] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:49.295727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963c309-8d78-40a1-a5d5-e4cc358aec56 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:49.297721 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963c309-8d78-40a1-a5d5-e4cc358aec56 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:49.298337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963c309-8d78-40a1-a5d5-e4cc358aec56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:49.339697 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-7963c309-8d78-40a1-a5d5-e4cc358aec56 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:49.341679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7963c309-8d78-40a1-a5d5-e4cc358aec56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:49.363289 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 945/1891] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:49.365128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66794425-c40f-4650-b8c6-ac0b2ef22ff6 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:49.398083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66794425-c40f-4650-b8c6-ac0b2ef22ff6 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:49.407415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b22ad-e880-4670-a950-26cafd6f5fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:49.408696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b22ad-e880-4670-a950-26cafd6f5fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:49.409203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b22ad-e880-4670-a950-26cafd6f5fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:49.409718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b22ad-e880-4670-a950-26cafd6f5fc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:49.411423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b22ad-e880-4670-a950-26cafd6f5fc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:10:49.412024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b22ad-e880-4670-a950-26cafd6f5fc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:49.415238 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b22ad-e880-4670-a950-26cafd6f5fc6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:49.442636 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 947/1892] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:10:49.449400 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963c309-8d78-40a1-a5d5-e4cc358aec56 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:49.467466 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7963c309-8d78-40a1-a5d5-e4cc358aec56 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:49.479485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07a88e-07ed-4c26-8630-73d9f6fabe23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:49.480724 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07a88e-07ed-4c26-8630-73d9f6fabe23 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:49.480724 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07a88e-07ed-4c26-8630-73d9f6fabe23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:49.481043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07a88e-07ed-4c26-8630-73d9f6fabe23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:49.482413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07a88e-07ed-4c26-8630-73d9f6fabe23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:49.486446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07a88e-07ed-4c26-8630-73d9f6fabe23 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:49.561647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b22ad-e880-4670-a950-26cafd6f5fc6 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:49.562130 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 946/1893] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:49 2022] POST /identity/v3/projects => generated 369 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:49.616508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b22ad-e880-4670-a950-26cafd6f5fc6 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:49.789248 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e929cd1-ed8a-4247-a981-a99eec07a473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:49.789903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e929cd1-ed8a-4247-a981-a99eec07a473 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:49.791272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e929cd1-ed8a-4247-a981-a99eec07a473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:49.791713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e929cd1-ed8a-4247-a981-a99eec07a473 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:49.792721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e929cd1-ed8a-4247-a981-a99eec07a473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:49.796795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e929cd1-ed8a-4247-a981-a99eec07a473 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:49.853648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64878c-a7f4-4a08-a62d-cfcc6cc498fc None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:49.856409 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64878c-a7f4-4a08-a62d-cfcc6cc498fc None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:49.860884 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64878c-a7f4-4a08-a62d-cfcc6cc498fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:49.871423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 948/1894] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:49 2022] POST /identity/v3/users => generated 285 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:49.879337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e929cd1-ed8a-4247-a981-a99eec07a473 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:49.942646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e929cd1-ed8a-4247-a981-a99eec07a473 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:49.967712 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b5f40-1edd-4734-8d74-4420cd54e7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:49.973021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b5f40-1edd-4734-8d74-4420cd54e7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:49.973021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b5f40-1edd-4734-8d74-4420cd54e7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:49.973021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b5f40-1edd-4734-8d74-4420cd54e7b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:49.981541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b5f40-1edd-4734-8d74-4420cd54e7b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:49.985008 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0b64878c-a7f4-4a08-a62d-cfcc6cc498fc None None] MFA Rules not processed for user `345df51fe411448fbd4b183233e899fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:49.986433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0b64878c-a7f4-4a08-a62d-cfcc6cc498fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:49.992162 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b5f40-1edd-4734-8d74-4420cd54e7b0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:50.063191 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 949/1895] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:50.064760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b5f40-1edd-4734-8d74-4420cd54e7b0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:50.069993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1b2b5f40-1edd-4734-8d74-4420cd54e7b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:50.162875 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b5f40-1edd-4734-8d74-4420cd54e7b0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:50.164962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:50.186230 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd91e60-cbad-42a1-889e-94d2ca894830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:50.188432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd91e60-cbad-42a1-889e-94d2ca894830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:50.188432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd91e60-cbad-42a1-889e-94d2ca894830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:50.188432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd91e60-cbad-42a1-889e-94d2ca894830 admin admin] PATH_INFO: `/v3/projects/5a06ead90e4f4477afbcf78794ac38b2` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:50.190621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 947/1896] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:10:50.193717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64878c-a7f4-4a08-a62d-cfcc6cc498fc None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:50.197322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd91e60-cbad-42a1-889e-94d2ca894830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a06ead90e4f4477afbcf78794ac38b2)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:50.201617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd91e60-cbad-42a1-889e-94d2ca894830 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:50.225525 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64878c-a7f4-4a08-a62d-cfcc6cc498fc None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:50.231723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7410c77-998f-468d-b471-410b37ea547e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:50.233109 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7410c77-998f-468d-b471-410b37ea547e admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:50.233501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7410c77-998f-468d-b471-410b37ea547e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:50.233907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7410c77-998f-468d-b471-410b37ea547e admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:50.234795 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7410c77-998f-468d-b471-410b37ea547e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:50.236791 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7410c77-998f-468d-b471-410b37ea547e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:50.257669 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 948/1897] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:50.258754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7410c77-998f-468d-b471-410b37ea547e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:50.271640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f7410c77-998f-468d-b471-410b37ea547e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:50.309679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7410c77-998f-468d-b471-410b37ea547e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:50.316348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61588e-8b9c-4a33-8907-1e73cc50a4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:50.317434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61588e-8b9c-4a33-8907-1e73cc50a4a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:50.319090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61588e-8b9c-4a33-8907-1e73cc50a4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:50.319852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61588e-8b9c-4a33-8907-1e73cc50a4a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:50.320857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61588e-8b9c-4a33-8907-1e73cc50a4a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:50.324186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61588e-8b9c-4a33-8907-1e73cc50a4a6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:50.401988 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0dd91e60-cbad-42a1-889e-94d2ca894830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a06ead90e4f4477afbcf78794ac38b2', 'request_id': 'req-0dd91e60-cbad-42a1-889e-94d2ca894830'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:50.407486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 949/1898] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:50 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:50.412722 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61588e-8b9c-4a33-8907-1e73cc50a4a6 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:50.428493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0dd91e60-cbad-42a1-889e-94d2ca894830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a06ead90e4f4477afbcf78794ac38b2', 'request_id': 'req-0dd91e60-cbad-42a1-889e-94d2ca894830'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:50.465421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0dd91e60-cbad-42a1-889e-94d2ca894830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a06ead90e4f4477afbcf78794ac38b2', 'request_id': 'req-0dd91e60-cbad-42a1-889e-94d2ca894830'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:50.524685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0dd91e60-cbad-42a1-889e-94d2ca894830 admin admin] The token cache is being invalidate because project 5a06ead90e4f4477afbcf78794ac38b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:10:50.525113 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0dd91e60-cbad-42a1-889e-94d2ca894830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:50.531148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0dd91e60-cbad-42a1-889e-94d2ca894830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:10:50.535273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 950/1899] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:10:50 2022] DELETE /identity/v3/projects/5a06ead90e4f4477afbcf78794ac38b2 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:50.536205 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd91e60-cbad-42a1-889e-94d2ca894830 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:50.541033 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0dd91e60-cbad-42a1-889e-94d2ca894830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:50.568740 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61588e-8b9c-4a33-8907-1e73cc50a4a6 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:50.592895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0c836-c2f2-4deb-a0af-c9d2e16d134f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:50.594774 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0c836-c2f2-4deb-a0af-c9d2e16d134f admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:50.599179 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0c836-c2f2-4deb-a0af-c9d2e16d134f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:50.599794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0c836-c2f2-4deb-a0af-c9d2e16d134f admin admin] PATH_INFO: `/v3/projects/d3149de22a83431fb4f39a957010c9b5/users/971ee571b6b94939a71dac9844c7e352/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:50.639038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0c836-c2f2-4deb-a0af-c9d2e16d134f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3149de22a83431fb4f39a957010c9b5, user_id=971ee571b6b94939a71dac9844c7e352, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:50.646794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0c836-c2f2-4deb-a0af-c9d2e16d134f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:50.670936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:50.705084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 950/1900] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:50 2022] PUT /identity/v3/projects/d3149de22a83431fb4f39a957010c9b5/users/971ee571b6b94939a71dac9844c7e352/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:50.705084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52c85e-ea7a-48a9-a8ac-ffc3c2c7f713 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:50.705084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52c85e-ea7a-48a9-a8ac-ffc3c2c7f713 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:50.705084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52c85e-ea7a-48a9-a8ac-ffc3c2c7f713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:50.706700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd91e60-cbad-42a1-889e-94d2ca894830 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:50.747166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-bb52c85e-ea7a-48a9-a8ac-ffc3c2c7f713 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:50.748956 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bb52c85e-ea7a-48a9-a8ac-ffc3c2c7f713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:50.838544 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22d1f1-a0fe-4e75-bf75-677bb9affc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:50.839756 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22d1f1-a0fe-4e75-bf75-677bb9affc94 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:50.840313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22d1f1-a0fe-4e75-bf75-677bb9affc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:50.841885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22d1f1-a0fe-4e75-bf75-677bb9affc94 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:50.842424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22d1f1-a0fe-4e75-bf75-677bb9affc94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:50.844074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 951/1901] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:50.846015 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b424b5-97e6-4732-aa77-44d6305c2445 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:50.846270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22d1f1-a0fe-4e75-bf75-677bb9affc94 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:50.847879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b424b5-97e6-4732-aa77-44d6305c2445 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:50.848814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b424b5-97e6-4732-aa77-44d6305c2445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:50.883554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 951/1902] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:50 2022] POST /identity/v3/users => generated 292 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:50.886419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1348b-f097-49e9-9647-ba8bc985ff5a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:50.886751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1348b-f097-49e9-9647-ba8bc985ff5a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:50.887165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1348b-f097-49e9-9647-ba8bc985ff5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:50.898889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-45b424b5-97e6-4732-aa77-44d6305c2445 None None] MFA Rules not processed for user `971ee571b6b94939a71dac9844c7e352`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:50.903168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-45b424b5-97e6-4732-aa77-44d6305c2445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:50.913969 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e0f1348b-f097-49e9-9647-ba8bc985ff5a None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:50.917117 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e0f1348b-f097-49e9-9647-ba8bc985ff5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:50.929562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 952/1903] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:50.930799 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1348b-f097-49e9-9647-ba8bc985ff5a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:50.945671 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1348b-f097-49e9-9647-ba8bc985ff5a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:50.952909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82727cf5-bda2-4b08-9713-ace7137f5d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:50.953963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82727cf5-bda2-4b08-9713-ace7137f5d47 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:50.954958 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82727cf5-bda2-4b08-9713-ace7137f5d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:50.955442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82727cf5-bda2-4b08-9713-ace7137f5d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:50.956798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82727cf5-bda2-4b08-9713-ace7137f5d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:50.959428 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82727cf5-bda2-4b08-9713-ace7137f5d47 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:50.973657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:50.979653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 953/1904] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:50.979653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82727cf5-bda2-4b08-9713-ace7137f5d47 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:50.991949 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82727cf5-bda2-4b08-9713-ace7137f5d47 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:51.002328 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745eb4a-aa3a-4cc1-b179-a17b0b7bf00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:51.003404 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745eb4a-aa3a-4cc1-b179-a17b0b7bf00c admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:51.003845 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745eb4a-aa3a-4cc1-b179-a17b0b7bf00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:51.004718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745eb4a-aa3a-4cc1-b179-a17b0b7bf00c admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:51.005798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745eb4a-aa3a-4cc1-b179-a17b0b7bf00c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:10:51.006506 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745eb4a-aa3a-4cc1-b179-a17b0b7bf00c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:51.008272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d745eb4a-aa3a-4cc1-b179-a17b0b7bf00c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:51.030134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 954/1905] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:10:51.030134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745eb4a-aa3a-4cc1-b179-a17b0b7bf00c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:51.043948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745eb4a-aa3a-4cc1-b179-a17b0b7bf00c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:51.049165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11ae1b-bea3-4f8e-b6bc-6adb54bf785e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:51.050240 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11ae1b-bea3-4f8e-b6bc-6adb54bf785e admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:51.050886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11ae1b-bea3-4f8e-b6bc-6adb54bf785e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:51.051264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11ae1b-bea3-4f8e-b6bc-6adb54bf785e admin admin] PATH_INFO: `/v3/projects/ee21144fc983451bbfa9731c6ec613af/users/127fc607363c47a9b0b34a07c6c189cb/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:51.075397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 952/1906] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:50 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:51.079099 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11ae1b-bea3-4f8e-b6bc-6adb54bf785e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee21144fc983451bbfa9731c6ec613af, user_id=127fc607363c47a9b0b34a07c6c189cb, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:51.079579 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b424b5-97e6-4732-aa77-44d6305c2445 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:51.081986 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11ae1b-bea3-4f8e-b6bc-6adb54bf785e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:51.103802 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b424b5-97e6-4732-aa77-44d6305c2445 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:51.121988 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 955/1907] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:51 2022] PUT /identity/v3/projects/ee21144fc983451bbfa9731c6ec613af/users/127fc607363c47a9b0b34a07c6c189cb/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:51.125309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11ae1b-bea3-4f8e-b6bc-6adb54bf785e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:51.143643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3c11ae1b-bea3-4f8e-b6bc-6adb54bf785e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:51.274203 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d094c5-1b2b-4213-b1d8-bdb4fa041e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:51.275096 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d094c5-1b2b-4213-b1d8-bdb4fa041e52 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:51.275867 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d094c5-1b2b-4213-b1d8-bdb4fa041e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:51.275867 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d094c5-1b2b-4213-b1d8-bdb4fa041e52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:51.276468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d094c5-1b2b-4213-b1d8-bdb4fa041e52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:51.283716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d094c5-1b2b-4213-b1d8-bdb4fa041e52 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:51.298208 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11ae1b-bea3-4f8e-b6bc-6adb54bf785e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:51.310263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89b87a-2b93-479b-801a-3451d6ef3764 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e053b6b25a4e4a20a05e6ffa6683222b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:51.311065 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89b87a-2b93-479b-801a-3451d6ef3764 service glance] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:51.311598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89b87a-2b93-479b-801a-3451d6ef3764 service glance] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:51.312260 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89b87a-2b93-479b-801a-3451d6ef3764 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:51.363388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 953/1908] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:51 2022] POST /identity/v3/projects => generated 409 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:51.366410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d094c5-1b2b-4213-b1d8-bdb4fa041e52 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:51.413947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:51.456680 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89b87a-2b93-479b-801a-3451d6ef3764 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:51.458360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89b87a-2b93-479b-801a-3451d6ef3764 service glance] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:51.493682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 956/1909] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:51 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:51.494951 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89b87a-2b93-479b-801a-3451d6ef3764 service glance] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:51.507828 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89b87a-2b93-479b-801a-3451d6ef3764 service glance] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:51.561044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d094c5-1b2b-4213-b1d8-bdb4fa041e52 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:51.596318 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e22b12-aa68-4fb7-98c8-bd5fe25087b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:51.597955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e22b12-aa68-4fb7-98c8-bd5fe25087b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:51.600089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e22b12-aa68-4fb7-98c8-bd5fe25087b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:51.600405 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e22b12-aa68-4fb7-98c8-bd5fe25087b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:51.601510 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e22b12-aa68-4fb7-98c8-bd5fe25087b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:51.605831 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e22b12-aa68-4fb7-98c8-bd5fe25087b0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:51.615124 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c60665-8354-4d63-8fc4-d39c95a509e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:51.615943 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c60665-8354-4d63-8fc4-d39c95a509e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:51.616259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c60665-8354-4d63-8fc4-d39c95a509e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:51.618935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c60665-8354-4d63-8fc4-d39c95a509e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:51.622999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c60665-8354-4d63-8fc4-d39c95a509e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:51.633169 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c60665-8354-4d63-8fc4-d39c95a509e2 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:51.650321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 954/1910] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:51.652629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e22b12-aa68-4fb7-98c8-bd5fe25087b0 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:51.666406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e22b12-aa68-4fb7-98c8-bd5fe25087b0 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:51.673646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105c22e-3b5b-4d06-8a2b-3a9aa692b4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:51.674821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105c22e-3b5b-4d06-8a2b-3a9aa692b4a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:51.675116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105c22e-3b5b-4d06-8a2b-3a9aa692b4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:51.675807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105c22e-3b5b-4d06-8a2b-3a9aa692b4a1 admin admin] PATH_INFO: `/v3/projects/ee21144fc983451bbfa9731c6ec613af/users/127fc607363c47a9b0b34a07c6c189cb/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:51.681953 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105c22e-3b5b-4d06-8a2b-3a9aa692b4a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee21144fc983451bbfa9731c6ec613af, user_id=127fc607363c47a9b0b34a07c6c189cb, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:51.681953 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105c22e-3b5b-4d06-8a2b-3a9aa692b4a1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:51.683224 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 957/1911] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:51 2022] POST /identity/v3/users => generated 305 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:51.685264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c60665-8354-4d63-8fc4-d39c95a509e2 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:51.700748 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 955/1912] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:51 2022] PUT /identity/v3/projects/ee21144fc983451bbfa9731c6ec613af/users/127fc607363c47a9b0b34a07c6c189cb/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:51.708116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c60665-8354-4d63-8fc4-d39c95a509e2 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:51.807295 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee5370-bb6d-49c7-b793-3feb5458fd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:51.808315 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee5370-bb6d-49c7-b793-3feb5458fd86 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:51.808771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee5370-bb6d-49c7-b793-3feb5458fd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:51.809184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee5370-bb6d-49c7-b793-3feb5458fd86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:51.810227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee5370-bb6d-49c7-b793-3feb5458fd86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:51.812206 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee5370-bb6d-49c7-b793-3feb5458fd86 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:51.838480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 958/1913] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:51.840275 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105c22e-3b5b-4d06-8a2b-3a9aa692b4a1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:51.854785 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105c22e-3b5b-4d06-8a2b-3a9aa692b4a1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:51.862809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425d274-cdbe-4d48-9e46-59d5f56c6c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:51.863868 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425d274-cdbe-4d48-9e46-59d5f56c6c0b admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:51.864141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425d274-cdbe-4d48-9e46-59d5f56c6c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:51.864530 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425d274-cdbe-4d48-9e46-59d5f56c6c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:51.865589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425d274-cdbe-4d48-9e46-59d5f56c6c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:51.867680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425d274-cdbe-4d48-9e46-59d5f56c6c0b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:51.890026 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 956/1914] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:51.894050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee5370-bb6d-49c7-b793-3feb5458fd86 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:51.916884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee5370-bb6d-49c7-b793-3feb5458fd86 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:51.927446 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869f963-6a40-42d5-a40b-91739ec93687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:51.928623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869f963-6a40-42d5-a40b-91739ec93687 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:51.929020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869f963-6a40-42d5-a40b-91739ec93687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:51.929613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869f963-6a40-42d5-a40b-91739ec93687 admin admin] PATH_INFO: `/v3/projects/79d34e2106a74d1d9257156278d6b8bf/users/434076f94431458cae12f6062bd1f03a/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:51.958773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869f963-6a40-42d5-a40b-91739ec93687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d34e2106a74d1d9257156278d6b8bf, user_id=434076f94431458cae12f6062bd1f03a, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:51.967141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869f963-6a40-42d5-a40b-91739ec93687 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:51.999981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 959/1915] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:51 2022] PUT /identity/v3/projects/79d34e2106a74d1d9257156278d6b8bf/users/434076f94431458cae12f6062bd1f03a/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:52.007633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d90d7f-1c0b-4198-8dca-11ce5214de2d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:52.008696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d90d7f-1c0b-4198-8dca-11ce5214de2d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:52.009256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d90d7f-1c0b-4198-8dca-11ce5214de2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:52.053956 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f0d90d7f-1c0b-4198-8dca-11ce5214de2d None None] MFA Rules not processed for user `434076f94431458cae12f6062bd1f03a`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:52.060385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f0d90d7f-1c0b-4198-8dca-11ce5214de2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:52.090651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:52.177136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 957/1916] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:52 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:52.182748 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869f963-6a40-42d5-a40b-91739ec93687 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:52.296444 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:52.343693 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869f963-6a40-42d5-a40b-91739ec93687 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:52.355117 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46830ce4-0fb3-4524-8a7b-fe9cf4e003eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:52.356613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46830ce4-0fb3-4524-8a7b-fe9cf4e003eb admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:52.357376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46830ce4-0fb3-4524-8a7b-fe9cf4e003eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:52.357936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46830ce4-0fb3-4524-8a7b-fe9cf4e003eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:52.359372 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46830ce4-0fb3-4524-8a7b-fe9cf4e003eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:52.364278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46830ce4-0fb3-4524-8a7b-fe9cf4e003eb admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:52.448938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d90d7f-1c0b-4198-8dca-11ce5214de2d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:52.460507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 960/1917] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:10:52 2022] POST /identity/v3/projects => generated 407 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:52.489061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d90d7f-1c0b-4198-8dca-11ce5214de2d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:52.768892 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0884dde-260d-4c58-9589-fdf38a0f72d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:52.770442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884dde-260d-4c58-9589-fdf38a0f72d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:52.770956 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884dde-260d-4c58-9589-fdf38a0f72d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:52.772150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884dde-260d-4c58-9589-fdf38a0f72d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:52.773666 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0884dde-260d-4c58-9589-fdf38a0f72d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:52.779111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0884dde-260d-4c58-9589-fdf38a0f72d6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:52.849750 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 958/1918] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:10:52 2022] POST /identity/v3/users => generated 310 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:10:52.854613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46830ce4-0fb3-4524-8a7b-fe9cf4e003eb admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:52.871236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46830ce4-0fb3-4524-8a7b-fe9cf4e003eb admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:52.876582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ead75-0586-4390-8c94-72b76697728c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:52.877393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ead75-0586-4390-8c94-72b76697728c admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:52.877748 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ead75-0586-4390-8c94-72b76697728c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:52.878102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ead75-0586-4390-8c94-72b76697728c admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:52.878965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ead75-0586-4390-8c94-72b76697728c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:52.881235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ead75-0586-4390-8c94-72b76697728c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:52.902556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 961/1919] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:52.906693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0884dde-260d-4c58-9589-fdf38a0f72d6 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:52.927663 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0884dde-260d-4c58-9589-fdf38a0f72d6 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:52.937010 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1023a0b-21f5-4663-b8a7-76cd4da19d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:52.938605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1023a0b-21f5-4663-b8a7-76cd4da19d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:52.939079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1023a0b-21f5-4663-b8a7-76cd4da19d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:52.939570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1023a0b-21f5-4663-b8a7-76cd4da19d7d admin admin] PATH_INFO: `/v3/projects/4470d4a35f694cbc97de8c5d682e5c00/users/971f1d0c21ca4dfa96c28b5134492db5/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:52.975655 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1023a0b-21f5-4663-b8a7-76cd4da19d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4470d4a35f694cbc97de8c5d682e5c00, user_id=971f1d0c21ca4dfa96c28b5134492db5, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:52.981017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1023a0b-21f5-4663-b8a7-76cd4da19d7d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:53.038605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 959/1920] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:52 2022] PUT /identity/v3/projects/4470d4a35f694cbc97de8c5d682e5c00/users/971f1d0c21ca4dfa96c28b5134492db5/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:53.044305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ead75-0586-4390-8c94-72b76697728c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:53.065643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ead75-0586-4390-8c94-72b76697728c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:53.260294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c798ac-981b-41d6-99e2-909e41cfca85 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:53.260978 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c798ac-981b-41d6-99e2-909e41cfca85 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:53.261610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c798ac-981b-41d6-99e2-909e41cfca85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:53.271678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db69d0-7800-40d6-b10b-8235b7b916ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:53.272884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db69d0-7800-40d6-b10b-8235b7b916ea admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:53.273355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db69d0-7800-40d6-b10b-8235b7b916ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:53.273834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db69d0-7800-40d6-b10b-8235b7b916ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:53.282705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db69d0-7800-40d6-b10b-8235b7b916ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:53.289654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db69d0-7800-40d6-b10b-8235b7b916ea admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:53.319759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-96c798ac-981b-41d6-99e2-909e41cfca85 None None] MFA Rules not processed for user `127fc607363c47a9b0b34a07c6c189cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:53.322237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-96c798ac-981b-41d6-99e2-909e41cfca85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:53.324134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 962/1921] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:10:53.330648 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db69d0-7800-40d6-b10b-8235b7b916ea admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:53.362265 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db69d0-7800-40d6-b10b-8235b7b916ea admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:53.375958 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec5d65-9b64-425e-ab62-1352533a6868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:53.378719 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec5d65-9b64-425e-ab62-1352533a6868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:53.379320 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec5d65-9b64-425e-ab62-1352533a6868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:53.380016 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec5d65-9b64-425e-ab62-1352533a6868 admin admin] PATH_INFO: `/v3/projects/4470d4a35f694cbc97de8c5d682e5c00/users/971f1d0c21ca4dfa96c28b5134492db5/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:53.391985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec5d65-9b64-425e-ab62-1352533a6868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4470d4a35f694cbc97de8c5d682e5c00, user_id=971f1d0c21ca4dfa96c28b5134492db5, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:53.399059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec5d65-9b64-425e-ab62-1352533a6868 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:53.441630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 963/1922] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:10:53 2022] PUT /identity/v3/projects/4470d4a35f694cbc97de8c5d682e5c00/users/971f1d0c21ca4dfa96c28b5134492db5/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:10:53.448352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357828bd-c330-4f2a-a213-72e8056723d7 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:53.449706 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357828bd-c330-4f2a-a213-72e8056723d7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:53.450163 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357828bd-c330-4f2a-a213-72e8056723d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:53.497899 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-357828bd-c330-4f2a-a213-72e8056723d7 None None] MFA Rules not processed for user `971f1d0c21ca4dfa96c28b5134492db5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:53.500095 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-357828bd-c330-4f2a-a213-72e8056723d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:53.502928 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 960/1923] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:10:53.522146 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c798ac-981b-41d6-99e2-909e41cfca85 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:53.526324 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-96c798ac-981b-41d6-99e2-909e41cfca85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:53.561180 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:53.655736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c798ac-981b-41d6-99e2-909e41cfca85 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:53.664967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 964/1924] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:10:53 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:53.687281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357828bd-c330-4f2a-a213-72e8056723d7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:53.691514 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-357828bd-c330-4f2a-a213-72e8056723d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:53.753115 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:10:53.759139 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942ff89-1da5-493f-ab19-5656c1d01383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:53.760969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942ff89-1da5-493f-ab19-5656c1d01383 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:53.761309 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942ff89-1da5-493f-ab19-5656c1d01383 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:53.762056 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942ff89-1da5-493f-ab19-5656c1d01383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:53.765550 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2942ff89-1da5-493f-ab19-5656c1d01383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:53.823856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357828bd-c330-4f2a-a213-72e8056723d7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:53.830215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e16c7-6f70-474b-9546-fc455be4b4ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:53.831223 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e16c7-6f70-474b-9546-fc455be4b4ba service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:53.831505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e16c7-6f70-474b-9546-fc455be4b4ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:53.831784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e16c7-6f70-474b-9546-fc455be4b4ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:53.846874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e16c7-6f70-474b-9546-fc455be4b4ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:53.848380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e16c7-6f70-474b-9546-fc455be4b4ba service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:53.868119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 965/1925] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:53 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:10:53.926611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942ff89-1da5-493f-ab19-5656c1d01383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:53.969967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942ff89-1da5-493f-ab19-5656c1d01383 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:54.018942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 961/1926] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:53 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:54.504960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e16c7-6f70-474b-9546-fc455be4b4ba service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:54.542480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e16c7-6f70-474b-9546-fc455be4b4ba service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:54.555414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825ee3e-faa6-4cc8-8285-2847a95b920c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:54.557158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825ee3e-faa6-4cc8-8285-2847a95b920c service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:54.557735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825ee3e-faa6-4cc8-8285-2847a95b920c service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:54.570515 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825ee3e-faa6-4cc8-8285-2847a95b920c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:54.718150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825ee3e-faa6-4cc8-8285-2847a95b920c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:54.722053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825ee3e-faa6-4cc8-8285-2847a95b920c service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:54.749006 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 966/1927] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:10:55.876102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c126e-79fb-4f3a-a2b1-9d39d2ccdc7b None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:55.877945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c126e-79fb-4f3a-a2b1-9d39d2ccdc7b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:55.878541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c126e-79fb-4f3a-a2b1-9d39d2ccdc7b None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:55.881845 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 962/1928] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jul 21 09:10:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 09:10:55.897992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa544947-0196-4bb7-8538-8426dae24a38 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:55.900492 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa544947-0196-4bb7-8538-8426dae24a38 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:55.902190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa544947-0196-4bb7-8538-8426dae24a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:55.957494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-aa544947-0196-4bb7-8538-8426dae24a38 None None] MFA Rules not processed for user `c48d41dcf9b344a8b2fec77f4dd75fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:10:55.958867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-aa544947-0196-4bb7-8538-8426dae24a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:56.095482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c126e-79fb-4f3a-a2b1-9d39d2ccdc7b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:56.121566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c126e-79fb-4f3a-a2b1-9d39d2ccdc7b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:56.126483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0a240-5ff7-47e2-a4ca-6d67845c8e0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:56.127488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0a240-5ff7-47e2-a4ca-6d67845c8e0a service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:56.132922 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0a240-5ff7-47e2-a4ca-6d67845c8e0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:56.133395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0a240-5ff7-47e2-a4ca-6d67845c8e0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:56.162082 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 967/1929] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jul 21 09:10:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:10:56.180639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa544947-0196-4bb7-8538-8426dae24a38 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:10:56.184192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-aa544947-0196-4bb7-8538-8426dae24a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:10:56.211134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa544947-0196-4bb7-8538-8426dae24a38 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:10:56.221594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6518a0-6f04-4838-88fc-97802cdaae5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:10:56.222637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6518a0-6f04-4838-88fc-97802cdaae5c service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:10:56.223111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6518a0-6f04-4838-88fc-97802cdaae5c service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:10:56.223956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6518a0-6f04-4838-88fc-97802cdaae5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:10:56.243700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6518a0-6f04-4838-88fc-97802cdaae5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:56.247548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6518a0-6f04-4838-88fc-97802cdaae5c service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:56.272941 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 968/1930] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:10:56.278424 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0a240-5ff7-47e2-a4ca-6d67845c8e0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:10:56.278424 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0a240-5ff7-47e2-a4ca-6d67845c8e0a service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:10:56.294892 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 963/1931] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:10:56 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:11:05.631397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6518a0-6f04-4838-88fc-97802cdaae5c service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:05.736032 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:11:05.780329 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6518a0-6f04-4838-88fc-97802cdaae5c service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:05.788456 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08b8a3-eacd-4974-885f-6dbdb9a9a810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:05.790063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08b8a3-eacd-4974-885f-6dbdb9a9a810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:05.790584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08b8a3-eacd-4974-885f-6dbdb9a9a810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:05.791094 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08b8a3-eacd-4974-885f-6dbdb9a9a810 admin admin] PATH_INFO: `/v3/users/434076f94431458cae12f6062bd1f03a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:05.793421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08b8a3-eacd-4974-885f-6dbdb9a9a810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=434076f94431458cae12f6062bd1f03a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:05.797507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08b8a3-eacd-4974-885f-6dbdb9a9a810 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:05.886662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7d08b8a3-eacd-4974-885f-6dbdb9a9a810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '434076f94431458cae12f6062bd1f03a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:05.887489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7d08b8a3-eacd-4974-885f-6dbdb9a9a810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '434076f94431458cae12f6062bd1f03a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:05.898255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7d08b8a3-eacd-4974-885f-6dbdb9a9a810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '434076f94431458cae12f6062bd1f03a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:05.924010 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7d08b8a3-eacd-4974-885f-6dbdb9a9a810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '434076f94431458cae12f6062bd1f03a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:05.935478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7d08b8a3-eacd-4974-885f-6dbdb9a9a810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '434076f94431458cae12f6062bd1f03a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:05.951841 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 969/1932] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:11:05 2022] DELETE /identity/v3/users/434076f94431458cae12f6062bd1f03a => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:11:05.958923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0e40-7943-4ea3-928a-e749aa4df1c2 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:05.959441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0e40-7943-4ea3-928a-e749aa4df1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:05.959988 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd0e40-7943-4ea3-928a-e749aa4df1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:05.990551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-8abd0e40-7943-4ea3-928a-e749aa4df1c2 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:11:05.992009 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8abd0e40-7943-4ea3-928a-e749aa4df1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:06.007000 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:11:06.110831 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 964/1933] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:11:06.125383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08b8a3-eacd-4974-885f-6dbdb9a9a810 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:06.126965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7d08b8a3-eacd-4974-885f-6dbdb9a9a810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:06.246511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08b8a3-eacd-4974-885f-6dbdb9a9a810 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:06.253098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c31682-02fc-41b6-81f7-1b8d4262e2d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:06.254237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c31682-02fc-41b6-81f7-1b8d4262e2d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:06.255525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c31682-02fc-41b6-81f7-1b8d4262e2d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:06.255525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c31682-02fc-41b6-81f7-1b8d4262e2d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:06.279425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c31682-02fc-41b6-81f7-1b8d4262e2d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:06.283179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c31682-02fc-41b6-81f7-1b8d4262e2d1 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:06.316013 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 970/1934] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:11:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:11:07.175804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd0e40-7943-4ea3-928a-e749aa4df1c2 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:07.178048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8abd0e40-7943-4ea3-928a-e749aa4df1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:07.198945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd0e40-7943-4ea3-928a-e749aa4df1c2 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:07.210577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96df82-d3e6-456c-bda3-f1a04a73e357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:07.211694 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96df82-d3e6-456c-bda3-f1a04a73e357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:07.212296 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96df82-d3e6-456c-bda3-f1a04a73e357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:07.212634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96df82-d3e6-456c-bda3-f1a04a73e357 admin admin] PATH_INFO: `/v3/projects/79d34e2106a74d1d9257156278d6b8bf` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:07.214190 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96df82-d3e6-456c-bda3-f1a04a73e357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d34e2106a74d1d9257156278d6b8bf)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:07.216563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96df82-d3e6-456c-bda3-f1a04a73e357 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:07.298408 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ea96df82-d3e6-456c-bda3-f1a04a73e357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d34e2106a74d1d9257156278d6b8bf', 'request_id': 'req-ea96df82-d3e6-456c-bda3-f1a04a73e357'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:07.312067 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ea96df82-d3e6-456c-bda3-f1a04a73e357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d34e2106a74d1d9257156278d6b8bf', 'request_id': 'req-ea96df82-d3e6-456c-bda3-f1a04a73e357'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:07.325692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ea96df82-d3e6-456c-bda3-f1a04a73e357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d34e2106a74d1d9257156278d6b8bf', 'request_id': 'req-ea96df82-d3e6-456c-bda3-f1a04a73e357'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:07.341253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ea96df82-d3e6-456c-bda3-f1a04a73e357 admin admin] The token cache is being invalidate because project 79d34e2106a74d1d9257156278d6b8bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:11:07.341615 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ea96df82-d3e6-456c-bda3-f1a04a73e357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:07.342447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ea96df82-d3e6-456c-bda3-f1a04a73e357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:07.344180 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 965/1935] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:11:07 2022] DELETE /identity/v3/projects/79d34e2106a74d1d9257156278d6b8bf => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:11:07.350327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c31682-02fc-41b6-81f7-1b8d4262e2d1 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:07.369339 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c31682-02fc-41b6-81f7-1b8d4262e2d1 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:07.468611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026c3e9-0ef8-4295-ba47-c2cbead4ff19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:07.469520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026c3e9-0ef8-4295-ba47-c2cbead4ff19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:07.469801 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026c3e9-0ef8-4295-ba47-c2cbead4ff19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:07.470074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026c3e9-0ef8-4295-ba47-c2cbead4ff19 admin admin] PATH_INFO: `/v3/users/971f1d0c21ca4dfa96c28b5134492db5` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:07.471502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026c3e9-0ef8-4295-ba47-c2cbead4ff19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=971f1d0c21ca4dfa96c28b5134492db5)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:07.473599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026c3e9-0ef8-4295-ba47-c2cbead4ff19 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:07.478732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:11:07.541144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c026c3e9-0ef8-4295-ba47-c2cbead4ff19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '971f1d0c21ca4dfa96c28b5134492db5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:07.541840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c026c3e9-0ef8-4295-ba47-c2cbead4ff19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '971f1d0c21ca4dfa96c28b5134492db5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:07.551862 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c026c3e9-0ef8-4295-ba47-c2cbead4ff19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '971f1d0c21ca4dfa96c28b5134492db5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:07.572483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c026c3e9-0ef8-4295-ba47-c2cbead4ff19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '971f1d0c21ca4dfa96c28b5134492db5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:07.580721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c026c3e9-0ef8-4295-ba47-c2cbead4ff19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '971f1d0c21ca4dfa96c28b5134492db5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:07.588061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 971/1936] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:11:07 2022] DELETE /identity/v3/users/971f1d0c21ca4dfa96c28b5134492db5 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:11:08.020127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96df82-d3e6-456c-bda3-f1a04a73e357 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:08.024767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ea96df82-d3e6-456c-bda3-f1a04a73e357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:08.039519 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96df82-d3e6-456c-bda3-f1a04a73e357 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:08.196956 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb34108-cf35-4615-90dc-28b0f1a54d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:08.198605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb34108-cf35-4615-90dc-28b0f1a54d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:08.199446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb34108-cf35-4615-90dc-28b0f1a54d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:08.199905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb34108-cf35-4615-90dc-28b0f1a54d6f admin admin] PATH_INFO: `/v3/projects/4470d4a35f694cbc97de8c5d682e5c00` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:08.201579 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb34108-cf35-4615-90dc-28b0f1a54d6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4470d4a35f694cbc97de8c5d682e5c00)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:08.204138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb34108-cf35-4615-90dc-28b0f1a54d6f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:08.265397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:11:08.280649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dbb34108-cf35-4615-90dc-28b0f1a54d6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4470d4a35f694cbc97de8c5d682e5c00', 'request_id': 'req-dbb34108-cf35-4615-90dc-28b0f1a54d6f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:08.311335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dbb34108-cf35-4615-90dc-28b0f1a54d6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4470d4a35f694cbc97de8c5d682e5c00', 'request_id': 'req-dbb34108-cf35-4615-90dc-28b0f1a54d6f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:08.323213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dbb34108-cf35-4615-90dc-28b0f1a54d6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4470d4a35f694cbc97de8c5d682e5c00', 'request_id': 'req-dbb34108-cf35-4615-90dc-28b0f1a54d6f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:08.349419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dbb34108-cf35-4615-90dc-28b0f1a54d6f admin admin] The token cache is being invalidate because project 4470d4a35f694cbc97de8c5d682e5c00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:11:08.349951 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dbb34108-cf35-4615-90dc-28b0f1a54d6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:08.350779 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dbb34108-cf35-4615-90dc-28b0f1a54d6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:08.353967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 966/1937] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:11:08 2022] DELETE /identity/v3/projects/4470d4a35f694cbc97de8c5d682e5c00 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:11:08.360790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b18d2-c79a-4f1b-a939-d61806d8c1b4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:08.361448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b18d2-c79a-4f1b-a939-d61806d8c1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:08.361615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b18d2-c79a-4f1b-a939-d61806d8c1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:08.408577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-2b8b18d2-c79a-4f1b-a939-d61806d8c1b4 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:11:08.416511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2b8b18d2-c79a-4f1b-a939-d61806d8c1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:08.537039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 972/1938] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:11:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:11:08.549852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3c87c-34a1-4369-b91b-6e5b37132086 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:08.550095 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3c87c-34a1-4369-b91b-6e5b37132086 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:08.550381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3c87c-34a1-4369-b91b-6e5b37132086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:08.577653 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c1b3c87c-34a1-4369-b91b-6e5b37132086 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:11:08.581259 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c1b3c87c-34a1-4369-b91b-6e5b37132086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:08.595707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 967/1939] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:11:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:11:08.597682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b18d2-c79a-4f1b-a939-d61806d8c1b4 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:08.613017 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b18d2-c79a-4f1b-a939-d61806d8c1b4 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:08.617882 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15401f8c-ca19-46fc-9b3b-78d56aa135aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:08.621306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15401f8c-ca19-46fc-9b3b-78d56aa135aa admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:08.621306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15401f8c-ca19-46fc-9b3b-78d56aa135aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:08.621306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15401f8c-ca19-46fc-9b3b-78d56aa135aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:08.621306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15401f8c-ca19-46fc-9b3b-78d56aa135aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:11:08.621306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15401f8c-ca19-46fc-9b3b-78d56aa135aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:08.622689 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15401f8c-ca19-46fc-9b3b-78d56aa135aa admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:08.641112 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 973/1940] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:11:08.646451 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3c87c-34a1-4369-b91b-6e5b37132086 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:08.668490 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3c87c-34a1-4369-b91b-6e5b37132086 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:08.678335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5826b-bccf-49e2-8d8f-6c18da1e5ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:08.679610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5826b-bccf-49e2-8d8f-6c18da1e5ebc admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:08.680251 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5826b-bccf-49e2-8d8f-6c18da1e5ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:08.681412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5826b-bccf-49e2-8d8f-6c18da1e5ebc admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:08.683257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5826b-bccf-49e2-8d8f-6c18da1e5ebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:08.687628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5826b-bccf-49e2-8d8f-6c18da1e5ebc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:08.752117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 968/1941] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:11:08 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:11:08.757642 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15401f8c-ca19-46fc-9b3b-78d56aa135aa admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:08.779464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15401f8c-ca19-46fc-9b3b-78d56aa135aa admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:08.870611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652c4bb-b2f2-4e9f-8e1b-513af70a9ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:08.871313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652c4bb-b2f2-4e9f-8e1b-513af70a9ebd admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:08.871597 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652c4bb-b2f2-4e9f-8e1b-513af70a9ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:08.871881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652c4bb-b2f2-4e9f-8e1b-513af70a9ebd admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:08.872630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652c4bb-b2f2-4e9f-8e1b-513af70a9ebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:08.874650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652c4bb-b2f2-4e9f-8e1b-513af70a9ebd admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:08.891471 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:11:08.901783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 974/1942] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:11:08 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:11:08.906843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5826b-bccf-49e2-8d8f-6c18da1e5ebc admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:08.918436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5826b-bccf-49e2-8d8f-6c18da1e5ebc admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:08.923680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f1751-55fa-4ec1-95bb-c4d096fff55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:08.924431 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f1751-55fa-4ec1-95bb-c4d096fff55d admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:08.924721 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f1751-55fa-4ec1-95bb-c4d096fff55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:08.924996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f1751-55fa-4ec1-95bb-c4d096fff55d admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:08.926032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f1751-55fa-4ec1-95bb-c4d096fff55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:08.927725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f1751-55fa-4ec1-95bb-c4d096fff55d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:08.940601 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 969/1943] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:11:08.946797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652c4bb-b2f2-4e9f-8e1b-513af70a9ebd admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:08.966735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652c4bb-b2f2-4e9f-8e1b-513af70a9ebd admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:08.974135 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd3417-a234-4b66-b143-db0ee4defe55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:08.975028 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd3417-a234-4b66-b143-db0ee4defe55 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:08.975382 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd3417-a234-4b66-b143-db0ee4defe55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:08.975885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd3417-a234-4b66-b143-db0ee4defe55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:08.977041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd3417-a234-4b66-b143-db0ee4defe55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:08.979416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd3417-a234-4b66-b143-db0ee4defe55 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:08.991630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 975/1944] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:11:08.996993 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f1751-55fa-4ec1-95bb-c4d096fff55d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:09.007717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f1751-55fa-4ec1-95bb-c4d096fff55d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:09.013151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c1e03-35b6-4ded-88ed-7ae9543f77c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:09.014062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c1e03-35b6-4ded-88ed-7ae9543f77c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:09.014344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c1e03-35b6-4ded-88ed-7ae9543f77c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:09.014618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c1e03-35b6-4ded-88ed-7ae9543f77c3 admin admin] PATH_INFO: `/v3/projects/725ea091bc4d473eb5b99bfafdf152f6/users/a97447dc4fb643a1b5d8f36e3fc99e38/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:09.032637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c1e03-35b6-4ded-88ed-7ae9543f77c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=725ea091bc4d473eb5b99bfafdf152f6, user_id=a97447dc4fb643a1b5d8f36e3fc99e38, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:09.035635 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c1e03-35b6-4ded-88ed-7ae9543f77c3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:09.053387 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 970/1945] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:11:08 2022] PUT /identity/v3/projects/725ea091bc4d473eb5b99bfafdf152f6/users/a97447dc4fb643a1b5d8f36e3fc99e38/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:11:09.064349 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9748d-d8c0-4ce4-8fb0-e82ac851154a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:09.064745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9748d-d8c0-4ce4-8fb0-e82ac851154a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:09.064887 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9748d-d8c0-4ce4-8fb0-e82ac851154a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:09.098160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e9c9748d-d8c0-4ce4-8fb0-e82ac851154a None None] MFA Rules not processed for user `a97447dc4fb643a1b5d8f36e3fc99e38`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:11:09.100403 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e9c9748d-d8c0-4ce4-8fb0-e82ac851154a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:09.222122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 976/1946] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:11:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:11:09.238804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c1e03-35b6-4ded-88ed-7ae9543f77c3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:09.241412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d18c1e03-35b6-4ded-88ed-7ae9543f77c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:09.322923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c1e03-35b6-4ded-88ed-7ae9543f77c3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:09.328351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf9582-e432-4bd6-bcc0-2bd22790c25a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:09.329629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf9582-e432-4bd6-bcc0-2bd22790c25a service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:09.329629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf9582-e432-4bd6-bcc0-2bd22790c25a service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:09.329893 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf9582-e432-4bd6-bcc0-2bd22790c25a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:09.342198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf9582-e432-4bd6-bcc0-2bd22790c25a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:09.343998 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf9582-e432-4bd6-bcc0-2bd22790c25a service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:09.360649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 971/1947] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:11:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:11:19.218230 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=38 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:11:19.352611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=62,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:11:20.975355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd657f-2d2b-4b3e-87a2-1af9821fbc7e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:20.976248 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd657f-2d2b-4b3e-87a2-1af9821fbc7e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:20.976421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd657f-2d2b-4b3e-87a2-1af9821fbc7e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:20.982783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 977/1948] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jul 21 09:11:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:11:21.002521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85721b74-aea6-4a38-91b8-134791153aac None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:21.003975 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85721b74-aea6-4a38-91b8-134791153aac None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:21.004667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85721b74-aea6-4a38-91b8-134791153aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:21.032060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-85721b74-aea6-4a38-91b8-134791153aac None None] MFA Rules not processed for user `72f9ef6de66e456e800237e9f954e98d`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:11:21.033426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-85721b74-aea6-4a38-91b8-134791153aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:21.139997 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 972/1949] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jul 21 09:11:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:11:21.177046 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd657f-2d2b-4b3e-87a2-1af9821fbc7e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:21.180407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c9bd657f-2d2b-4b3e-87a2-1af9821fbc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:21.210304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd657f-2d2b-4b3e-87a2-1af9821fbc7e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:21.223691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78030b6-f9c8-4cb6-9b78-60dfa23c8c6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:21.225063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78030b6-f9c8-4cb6-9b78-60dfa23c8c6b service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:21.225677 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78030b6-f9c8-4cb6-9b78-60dfa23c8c6b service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:21.227298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78030b6-f9c8-4cb6-9b78-60dfa23c8c6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:21.247639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78030b6-f9c8-4cb6-9b78-60dfa23c8c6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:21.250803 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78030b6-f9c8-4cb6-9b78-60dfa23c8c6b service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:21.268121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 978/1950] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:11:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:11:22.140704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d45698-daa6-4f20-aff6-f72c5ec7f3d4 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:22.141594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d45698-daa6-4f20-aff6-f72c5ec7f3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:22.141759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d45698-daa6-4f20-aff6-f72c5ec7f3d4 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:22.143761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 973/1951] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jul 21 09:11:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:11:22.216441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e216d-027e-4f0b-aa5c-0509a97862d6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:22.216991 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e216d-027e-4f0b-aa5c-0509a97862d6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:22.217520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e216d-027e-4f0b-aa5c-0509a97862d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:22.250693 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b21e216d-027e-4f0b-aa5c-0509a97862d6 None None] MFA Rules not processed for user `72f9ef6de66e456e800237e9f954e98d`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:11:22.254489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b21e216d-027e-4f0b-aa5c-0509a97862d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:22.272497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 979/1952] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jul 21 09:11:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:11:22.299575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d45698-daa6-4f20-aff6-f72c5ec7f3d4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:22.312775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e216d-027e-4f0b-aa5c-0509a97862d6 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:22.320142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b21e216d-027e-4f0b-aa5c-0509a97862d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:22.331825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d45698-daa6-4f20-aff6-f72c5ec7f3d4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:22.342058 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fafb2-689e-48af-8dff-7064cebfbbe6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:22.343175 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fafb2-689e-48af-8dff-7064cebfbbe6 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:22.343687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fafb2-689e-48af-8dff-7064cebfbbe6 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:22.344248 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fafb2-689e-48af-8dff-7064cebfbbe6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:22.359523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fafb2-689e-48af-8dff-7064cebfbbe6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:22.361282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fafb2-689e-48af-8dff-7064cebfbbe6 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:22.378080 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 974/1953] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:11:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:11:22.464251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e216d-027e-4f0b-aa5c-0509a97862d6 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:22.472656 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bab590-4574-4f78-89fa-686738ba8ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:22.474416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bab590-4574-4f78-89fa-686738ba8ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:22.475123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bab590-4574-4f78-89fa-686738ba8ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:22.475625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bab590-4574-4f78-89fa-686738ba8ac3 admin admin] PATH_INFO: `/v3/users/345df51fe411448fbd4b183233e899fd` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:22.478793 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bab590-4574-4f78-89fa-686738ba8ac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=345df51fe411448fbd4b183233e899fd)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:22.484278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bab590-4574-4f78-89fa-686738ba8ac3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:22.596078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e6bab590-4574-4f78-89fa-686738ba8ac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '345df51fe411448fbd4b183233e899fd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:22.597184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e6bab590-4574-4f78-89fa-686738ba8ac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '345df51fe411448fbd4b183233e899fd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:22.611219 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e6bab590-4574-4f78-89fa-686738ba8ac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '345df51fe411448fbd4b183233e899fd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:22.629047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e6bab590-4574-4f78-89fa-686738ba8ac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '345df51fe411448fbd4b183233e899fd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:22.643370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e6bab590-4574-4f78-89fa-686738ba8ac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '345df51fe411448fbd4b183233e899fd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:22.650981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 980/1954] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:11:22 2022] DELETE /identity/v3/users/345df51fe411448fbd4b183233e899fd => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:11:22.933034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fafb2-689e-48af-8dff-7064cebfbbe6 service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:22.958799 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fafb2-689e-48af-8dff-7064cebfbbe6 service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:23.088475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c15c30-be8b-476e-8249-40ab9ee60583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:23.089465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c15c30-be8b-476e-8249-40ab9ee60583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:23.089824 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c15c30-be8b-476e-8249-40ab9ee60583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:23.090183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c15c30-be8b-476e-8249-40ab9ee60583 admin admin] PATH_INFO: `/v3/projects/9a23621f15104d378cb86f0b3914082e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:23.091867 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c15c30-be8b-476e-8249-40ab9ee60583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a23621f15104d378cb86f0b3914082e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:23.094399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c15c30-be8b-476e-8249-40ab9ee60583 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:23.184634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e0c15c30-be8b-476e-8249-40ab9ee60583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a23621f15104d378cb86f0b3914082e', 'request_id': 'req-e0c15c30-be8b-476e-8249-40ab9ee60583'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:23.199830 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e0c15c30-be8b-476e-8249-40ab9ee60583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a23621f15104d378cb86f0b3914082e', 'request_id': 'req-e0c15c30-be8b-476e-8249-40ab9ee60583'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:23.212186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e0c15c30-be8b-476e-8249-40ab9ee60583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a23621f15104d378cb86f0b3914082e', 'request_id': 'req-e0c15c30-be8b-476e-8249-40ab9ee60583'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:23.239239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e0c15c30-be8b-476e-8249-40ab9ee60583 admin admin] The token cache is being invalidate because project 9a23621f15104d378cb86f0b3914082e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:11:23.240164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e0c15c30-be8b-476e-8249-40ab9ee60583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:23.241756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e0c15c30-be8b-476e-8249-40ab9ee60583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:23.244957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 975/1955] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:11:22 2022] DELETE /identity/v3/projects/9a23621f15104d378cb86f0b3914082e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:11:23.249270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bab590-4574-4f78-89fa-686738ba8ac3 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:23.250859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e6bab590-4574-4f78-89fa-686738ba8ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:23.266588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bab590-4574-4f78-89fa-686738ba8ac3 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:23.333736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123f48b-1c3e-4834-b797-1e61a45277d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:23.334540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123f48b-1c3e-4834-b797-1e61a45277d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:23.334829 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123f48b-1c3e-4834-b797-1e61a45277d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:23.335109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123f48b-1c3e-4834-b797-1e61a45277d4 admin admin] PATH_INFO: `/v3/users/127fc607363c47a9b0b34a07c6c189cb` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:23.337132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123f48b-1c3e-4834-b797-1e61a45277d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=127fc607363c47a9b0b34a07c6c189cb)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:23.339323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123f48b-1c3e-4834-b797-1e61a45277d4 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:23.341273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:11:23.395326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4123f48b-1c3e-4834-b797-1e61a45277d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127fc607363c47a9b0b34a07c6c189cb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:23.396045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4123f48b-1c3e-4834-b797-1e61a45277d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127fc607363c47a9b0b34a07c6c189cb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:23.404904 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4123f48b-1c3e-4834-b797-1e61a45277d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127fc607363c47a9b0b34a07c6c189cb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:23.419631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4123f48b-1c3e-4834-b797-1e61a45277d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127fc607363c47a9b0b34a07c6c189cb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:23.427386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4123f48b-1c3e-4834-b797-1e61a45277d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127fc607363c47a9b0b34a07c6c189cb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:23.434564 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 981/1956] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:11:23 2022] DELETE /identity/v3/users/127fc607363c47a9b0b34a07c6c189cb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:11:23.771141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c15c30-be8b-476e-8249-40ab9ee60583 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:23.772929 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e0c15c30-be8b-476e-8249-40ab9ee60583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:23.811018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c15c30-be8b-476e-8249-40ab9ee60583 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:23.835168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:11:23.940734 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8eba8f-cdf9-4cb2-b805-b5df22b817a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:23.941609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8eba8f-cdf9-4cb2-b805-b5df22b817a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:23.941895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8eba8f-cdf9-4cb2-b805-b5df22b817a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:23.942175 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8eba8f-cdf9-4cb2-b805-b5df22b817a3 admin admin] PATH_INFO: `/v3/projects/ee21144fc983451bbfa9731c6ec613af` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:23.943521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8eba8f-cdf9-4cb2-b805-b5df22b817a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee21144fc983451bbfa9731c6ec613af)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:23.945727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8eba8f-cdf9-4cb2-b805-b5df22b817a3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:24.007015 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8f8eba8f-cdf9-4cb2-b805-b5df22b817a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee21144fc983451bbfa9731c6ec613af', 'request_id': 'req-8f8eba8f-cdf9-4cb2-b805-b5df22b817a3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:24.015390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8f8eba8f-cdf9-4cb2-b805-b5df22b817a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee21144fc983451bbfa9731c6ec613af', 'request_id': 'req-8f8eba8f-cdf9-4cb2-b805-b5df22b817a3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:24.026538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8f8eba8f-cdf9-4cb2-b805-b5df22b817a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee21144fc983451bbfa9731c6ec613af', 'request_id': 'req-8f8eba8f-cdf9-4cb2-b805-b5df22b817a3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:24.044615 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8f8eba8f-cdf9-4cb2-b805-b5df22b817a3 admin admin] The token cache is being invalidate because project ee21144fc983451bbfa9731c6ec613af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:11:24.044912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8f8eba8f-cdf9-4cb2-b805-b5df22b817a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:24.045758 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8f8eba8f-cdf9-4cb2-b805-b5df22b817a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:24.053044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 976/1957] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:11:23 2022] DELETE /identity/v3/projects/ee21144fc983451bbfa9731c6ec613af => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:11:24.055895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9751b-c8ad-418c-a27e-25d62472b9a5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:24.056426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9751b-c8ad-418c-a27e-25d62472b9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:24.056722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9751b-c8ad-418c-a27e-25d62472b9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:24.085998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-41e9751b-c8ad-418c-a27e-25d62472b9a5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:11:24.087369 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-41e9751b-c8ad-418c-a27e-25d62472b9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:24.196759 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 982/1958] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:11:24.210181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e6121-710a-468f-89f4-1266c532f667 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:24.211011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e6121-710a-468f-89f4-1266c532f667 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:24.211771 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e6121-710a-468f-89f4-1266c532f667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:24.237374 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6e3e6121-710a-468f-89f4-1266c532f667 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:11:24.238829 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6e3e6121-710a-468f-89f4-1266c532f667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:24.251286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 977/1959] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:11:24.259203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e9751b-c8ad-418c-a27e-25d62472b9a5 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:24.278667 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e9751b-c8ad-418c-a27e-25d62472b9a5 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:24.284536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dda32-1a2d-4e0e-8def-e4f5c1b7bd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:24.288466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dda32-1a2d-4e0e-8def-e4f5c1b7bd7e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:24.289103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dda32-1a2d-4e0e-8def-e4f5c1b7bd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:24.290460 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dda32-1a2d-4e0e-8def-e4f5c1b7bd7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:24.292164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91dda32-1a2d-4e0e-8def-e4f5c1b7bd7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:11:24.292727 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91dda32-1a2d-4e0e-8def-e4f5c1b7bd7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:24.296298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91dda32-1a2d-4e0e-8def-e4f5c1b7bd7e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:24.321484 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 983/1960] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:11:24.324153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e6121-710a-468f-89f4-1266c532f667 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:24.345902 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e6121-710a-468f-89f4-1266c532f667 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:24.355173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a66cdb-00a0-4091-bfb6-ba8156e72281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:24.356492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a66cdb-00a0-4091-bfb6-ba8156e72281 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:24.357082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a66cdb-00a0-4091-bfb6-ba8156e72281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:24.357881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a66cdb-00a0-4091-bfb6-ba8156e72281 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:24.359092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a66cdb-00a0-4091-bfb6-ba8156e72281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:24.362380 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a66cdb-00a0-4091-bfb6-ba8156e72281 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:24.404805 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 978/1961] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:11:24 2022] POST /identity/v3/projects => generated 385 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:11:24.409433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dda32-1a2d-4e0e-8def-e4f5c1b7bd7e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:24.421221 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dda32-1a2d-4e0e-8def-e4f5c1b7bd7e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:24.515188 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae94a2-cce2-42c4-9282-537cee2f7bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:24.516152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae94a2-cce2-42c4-9282-537cee2f7bfa admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:24.516434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae94a2-cce2-42c4-9282-537cee2f7bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:24.516743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae94a2-cce2-42c4-9282-537cee2f7bfa admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:24.517521 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae94a2-cce2-42c4-9282-537cee2f7bfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:24.519536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae94a2-cce2-42c4-9282-537cee2f7bfa admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:24.535774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:11:24.563108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 984/1962] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:11:24 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:11:24.564317 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a66cdb-00a0-4091-bfb6-ba8156e72281 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:24.582744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a66cdb-00a0-4091-bfb6-ba8156e72281 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:24.592870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45beace-e432-40fe-a6a7-5e3f3272df50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:24.594381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45beace-e432-40fe-a6a7-5e3f3272df50 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:24.595057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45beace-e432-40fe-a6a7-5e3f3272df50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:24.595734 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45beace-e432-40fe-a6a7-5e3f3272df50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:24.597261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45beace-e432-40fe-a6a7-5e3f3272df50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:24.600308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45beace-e432-40fe-a6a7-5e3f3272df50 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:24.625737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 979/1963] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:11:24.630111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae94a2-cce2-42c4-9282-537cee2f7bfa admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:24.649491 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae94a2-cce2-42c4-9282-537cee2f7bfa admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:24.655002 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f3b37-e0d0-4197-a123-761776e79e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:24.655696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f3b37-e0d0-4197-a123-761776e79e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:24.656027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f3b37-e0d0-4197-a123-761776e79e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:24.656310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f3b37-e0d0-4197-a123-761776e79e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:24.657053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f3b37-e0d0-4197-a123-761776e79e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:24.658715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f3b37-e0d0-4197-a123-761776e79e12 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:24.672113 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 985/1964] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:11:24.679068 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45beace-e432-40fe-a6a7-5e3f3272df50 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:24.697103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45beace-e432-40fe-a6a7-5e3f3272df50 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:24.704964 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aada0d-8a9d-40ae-aa37-7a2dd1823bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:24.706616 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aada0d-8a9d-40ae-aa37-7a2dd1823bd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:24.707132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aada0d-8a9d-40ae-aa37-7a2dd1823bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:24.707636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aada0d-8a9d-40ae-aa37-7a2dd1823bd6 admin admin] PATH_INFO: `/v3/projects/8878ee3f2e0b42139df5a57557e171be/users/fd7ce4ccf52c4d219f85bbc3e59d1bf6/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:24.733957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aada0d-8a9d-40ae-aa37-7a2dd1823bd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8878ee3f2e0b42139df5a57557e171be, user_id=fd7ce4ccf52c4d219f85bbc3e59d1bf6, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:24.739416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aada0d-8a9d-40ae-aa37-7a2dd1823bd6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:24.759564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 980/1965] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:11:24 2022] PUT /identity/v3/projects/8878ee3f2e0b42139df5a57557e171be/users/fd7ce4ccf52c4d219f85bbc3e59d1bf6/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:11:24.771225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e0cfc-c9eb-44af-8b2d-af4674c1d103 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:24.771547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e0cfc-c9eb-44af-8b2d-af4674c1d103 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:24.771836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e0cfc-c9eb-44af-8b2d-af4674c1d103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:24.809116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ed6e0cfc-c9eb-44af-8b2d-af4674c1d103 None None] MFA Rules not processed for user `fd7ce4ccf52c4d219f85bbc3e59d1bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:11:24.810724 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ed6e0cfc-c9eb-44af-8b2d-af4674c1d103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:24.890347 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 986/1966] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:11:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:11:24.905313 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aada0d-8a9d-40ae-aa37-7a2dd1823bd6 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:24.907983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-44aada0d-8a9d-40ae-aa37-7a2dd1823bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:24.997518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:11:25.021904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aada0d-8a9d-40ae-aa37-7a2dd1823bd6 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:25.026872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee684a83-d5d3-4b7f-8c4d-07e7ece288d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:25.027527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee684a83-d5d3-4b7f-8c4d-07e7ece288d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:25.027810 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee684a83-d5d3-4b7f-8c4d-07e7ece288d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:25.028124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee684a83-d5d3-4b7f-8c4d-07e7ece288d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:25.038743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee684a83-d5d3-4b7f-8c4d-07e7ece288d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:25.042090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee684a83-d5d3-4b7f-8c4d-07e7ece288d1 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:25.053393 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 981/1967] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:11:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:11:25.076518 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e0cfc-c9eb-44af-8b2d-af4674c1d103 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:25.165526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e0cfc-c9eb-44af-8b2d-af4674c1d103 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:25.170395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ededf-1365-472e-bbe0-45ec06de4682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:25.171309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ededf-1365-472e-bbe0-45ec06de4682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:25.171604 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ededf-1365-472e-bbe0-45ec06de4682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:25.171876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ededf-1365-472e-bbe0-45ec06de4682 admin admin] PATH_INFO: `/v3/users/fd7ce4ccf52c4d219f85bbc3e59d1bf6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:25.173265 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ededf-1365-472e-bbe0-45ec06de4682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd7ce4ccf52c4d219f85bbc3e59d1bf6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:25.175271 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ededf-1365-472e-bbe0-45ec06de4682 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:25.249771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-328ededf-1365-472e-bbe0-45ec06de4682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd7ce4ccf52c4d219f85bbc3e59d1bf6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:25.250589 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-328ededf-1365-472e-bbe0-45ec06de4682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd7ce4ccf52c4d219f85bbc3e59d1bf6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:25.263398 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-328ededf-1365-472e-bbe0-45ec06de4682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd7ce4ccf52c4d219f85bbc3e59d1bf6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:25.285401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-328ededf-1365-472e-bbe0-45ec06de4682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd7ce4ccf52c4d219f85bbc3e59d1bf6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:25.300914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:11:25.314919 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-328ededf-1365-472e-bbe0-45ec06de4682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd7ce4ccf52c4d219f85bbc3e59d1bf6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:25.329558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 987/1968] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:11:25 2022] DELETE /identity/v3/users/fd7ce4ccf52c4d219f85bbc3e59d1bf6 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:11:25.337673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c626f-6054-40b9-8b7e-19d814ee98a8 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:25.338590 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c626f-6054-40b9-8b7e-19d814ee98a8 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:25.338590 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c626f-6054-40b9-8b7e-19d814ee98a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:25.364593 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-bf7c626f-6054-40b9-8b7e-19d814ee98a8 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:11:25.366736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bf7c626f-6054-40b9-8b7e-19d814ee98a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:25.471121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 982/1969] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:11:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:11:25.481628 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ededf-1365-472e-bbe0-45ec06de4682 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:25.484296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-328ededf-1365-472e-bbe0-45ec06de4682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:25.582339 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ededf-1365-472e-bbe0-45ec06de4682 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:25.586870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197273d7-2adb-4c90-8d04-906852638683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:25.590198 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197273d7-2adb-4c90-8d04-906852638683 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:25.590480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197273d7-2adb-4c90-8d04-906852638683 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:25.590754 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197273d7-2adb-4c90-8d04-906852638683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:25.606789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197273d7-2adb-4c90-8d04-906852638683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:25.608409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197273d7-2adb-4c90-8d04-906852638683 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:25.620715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 988/1970] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:11:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:11:26.200383 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c626f-6054-40b9-8b7e-19d814ee98a8 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:26.202647 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bf7c626f-6054-40b9-8b7e-19d814ee98a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:26.229930 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c626f-6054-40b9-8b7e-19d814ee98a8 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:26.241130 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9baac9a-ec57-4103-a3d5-3f6911974b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:26.243103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9baac9a-ec57-4103-a3d5-3f6911974b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:26.243624 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9baac9a-ec57-4103-a3d5-3f6911974b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:26.244165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9baac9a-ec57-4103-a3d5-3f6911974b01 admin admin] PATH_INFO: `/v3/projects/8878ee3f2e0b42139df5a57557e171be` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:26.246526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9baac9a-ec57-4103-a3d5-3f6911974b01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8878ee3f2e0b42139df5a57557e171be)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:26.251144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9baac9a-ec57-4103-a3d5-3f6911974b01 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:26.317819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c9baac9a-ec57-4103-a3d5-3f6911974b01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8878ee3f2e0b42139df5a57557e171be', 'request_id': 'req-c9baac9a-ec57-4103-a3d5-3f6911974b01'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:26.327122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c9baac9a-ec57-4103-a3d5-3f6911974b01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8878ee3f2e0b42139df5a57557e171be', 'request_id': 'req-c9baac9a-ec57-4103-a3d5-3f6911974b01'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:26.340140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c9baac9a-ec57-4103-a3d5-3f6911974b01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8878ee3f2e0b42139df5a57557e171be', 'request_id': 'req-c9baac9a-ec57-4103-a3d5-3f6911974b01'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:26.357663 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c9baac9a-ec57-4103-a3d5-3f6911974b01 admin admin] The token cache is being invalidate because project 8878ee3f2e0b42139df5a57557e171be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:11:26.357942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c9baac9a-ec57-4103-a3d5-3f6911974b01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:26.358617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c9baac9a-ec57-4103-a3d5-3f6911974b01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:11:26.364494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 983/1971] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:11:26 2022] DELETE /identity/v3/projects/8878ee3f2e0b42139df5a57557e171be => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:11:26.367433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b15ef-0ed3-412c-88ef-5e9c51f68713 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:26.368183 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b15ef-0ed3-412c-88ef-5e9c51f68713 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:26.368681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b15ef-0ed3-412c-88ef-5e9c51f68713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:26.399240 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3d4b15ef-0ed3-412c-88ef-5e9c51f68713 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:11:26.401925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3d4b15ef-0ed3-412c-88ef-5e9c51f68713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:26.478964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 989/1972] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:11:26.488167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357f458-35e0-41da-9ff1-0579425047f0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:26.488679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357f458-35e0-41da-9ff1-0579425047f0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:26.489284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0357f458-35e0-41da-9ff1-0579425047f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:26.521462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0357f458-35e0-41da-9ff1-0579425047f0 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:11:26.523662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0357f458-35e0-41da-9ff1-0579425047f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:26.538507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 984/1973] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:11:26.543525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b15ef-0ed3-412c-88ef-5e9c51f68713 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:26.554729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b15ef-0ed3-412c-88ef-5e9c51f68713 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:26.560591 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38862d-72b3-4069-8175-44d60c004b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:26.566276 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38862d-72b3-4069-8175-44d60c004b59 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:26.566794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38862d-72b3-4069-8175-44d60c004b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:26.567327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38862d-72b3-4069-8175-44d60c004b59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:26.568866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b38862d-72b3-4069-8175-44d60c004b59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:11:26.569443 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b38862d-72b3-4069-8175-44d60c004b59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:26.573215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b38862d-72b3-4069-8175-44d60c004b59 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:26.593716 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 990/1974] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:11:26.599903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357f458-35e0-41da-9ff1-0579425047f0 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:26.614267 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0357f458-35e0-41da-9ff1-0579425047f0 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:26.621609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8847c0-17d9-46a5-aa0e-4cd4f934cd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:26.623485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8847c0-17d9-46a5-aa0e-4cd4f934cd91 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:26.623485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8847c0-17d9-46a5-aa0e-4cd4f934cd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:26.623485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8847c0-17d9-46a5-aa0e-4cd4f934cd91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:26.624547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8847c0-17d9-46a5-aa0e-4cd4f934cd91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:26.627141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8847c0-17d9-46a5-aa0e-4cd4f934cd91 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:26.679519 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 985/1975] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:11:26 2022] POST /identity/v3/projects => generated 365 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:11:26.690870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38862d-72b3-4069-8175-44d60c004b59 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:26.702332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38862d-72b3-4069-8175-44d60c004b59 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:26.785758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161bca8-a00c-40f9-a1a3-d0d8300a370c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:26.786448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161bca8-a00c-40f9-a1a3-d0d8300a370c admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:26.786731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161bca8-a00c-40f9-a1a3-d0d8300a370c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:26.787009 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161bca8-a00c-40f9-a1a3-d0d8300a370c admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:26.787723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161bca8-a00c-40f9-a1a3-d0d8300a370c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:26.789917 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161bca8-a00c-40f9-a1a3-d0d8300a370c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:26.822063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 991/1976] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:11:26 2022] POST /identity/v3/users => generated 283 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:11:26.823517 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8847c0-17d9-46a5-aa0e-4cd4f934cd91 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:26.835371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8847c0-17d9-46a5-aa0e-4cd4f934cd91 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:26.850134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955bbe3-45c6-408c-9786-51440466e7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:26.851269 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955bbe3-45c6-408c-9786-51440466e7ba admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:26.852881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955bbe3-45c6-408c-9786-51440466e7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:26.853531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955bbe3-45c6-408c-9786-51440466e7ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:26.855071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955bbe3-45c6-408c-9786-51440466e7ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:26.858337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955bbe3-45c6-408c-9786-51440466e7ba admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:26.891833 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 986/1977] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:11:26.896727 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161bca8-a00c-40f9-a1a3-d0d8300a370c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:26.919253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161bca8-a00c-40f9-a1a3-d0d8300a370c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:26.924422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75f425-8d1c-4df0-bf1d-afd320e8867b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:26.925104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75f425-8d1c-4df0-bf1d-afd320e8867b admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:26.925669 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75f425-8d1c-4df0-bf1d-afd320e8867b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:26.926154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75f425-8d1c-4df0-bf1d-afd320e8867b admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:26.927318 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75f425-8d1c-4df0-bf1d-afd320e8867b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:26.929597 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75f425-8d1c-4df0-bf1d-afd320e8867b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:26.952204 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 992/1978] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:11:26.954764 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955bbe3-45c6-408c-9786-51440466e7ba admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:26.972702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955bbe3-45c6-408c-9786-51440466e7ba admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:26.979040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f01b9-df80-4543-b71e-491d1e8b9fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:26.980071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f01b9-df80-4543-b71e-491d1e8b9fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:26.980383 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f01b9-df80-4543-b71e-491d1e8b9fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:26.980662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f01b9-df80-4543-b71e-491d1e8b9fa3 admin admin] PATH_INFO: `/v3/projects/7237d9a716224e1aac9f1277211fc998/users/5408e0c011514976a6d7b51823deb563/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:26.999108 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f01b9-df80-4543-b71e-491d1e8b9fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7237d9a716224e1aac9f1277211fc998, user_id=5408e0c011514976a6d7b51823deb563, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:27.001855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f01b9-df80-4543-b71e-491d1e8b9fa3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:27.019070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 987/1979] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:11:26 2022] PUT /identity/v3/projects/7237d9a716224e1aac9f1277211fc998/users/5408e0c011514976a6d7b51823deb563/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:11:27.031803 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f16fd-9d26-4d39-a6af-7277aaa5bbeb None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:27.032530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f16fd-9d26-4d39-a6af-7277aaa5bbeb None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:27.033140 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f16fd-9d26-4d39-a6af-7277aaa5bbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:27.041564 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:11:27.074596 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0a0f16fd-9d26-4d39-a6af-7277aaa5bbeb None None] MFA Rules not processed for user `5408e0c011514976a6d7b51823deb563`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:11:27.077589 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0a0f16fd-9d26-4d39-a6af-7277aaa5bbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:27.237512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 993/1980] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:11:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:11:27.257527 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f01b9-df80-4543-b71e-491d1e8b9fa3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:11:27.262239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-173f01b9-df80-4543-b71e-491d1e8b9fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:11:27.272839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:11:27.381308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f01b9-df80-4543-b71e-491d1e8b9fa3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:11:27.386562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d8418-9d91-4650-9fe5-10685d236f04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:11:27.388774 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d8418-9d91-4650-9fe5-10685d236f04 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:11:27.388774 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d8418-9d91-4650-9fe5-10685d236f04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:11:27.388774 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d8418-9d91-4650-9fe5-10685d236f04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:11:27.405855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d8418-9d91-4650-9fe5-10685d236f04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:11:27.409801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d8418-9d91-4650-9fe5-10685d236f04 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:11:27.426933 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 988/1981] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:11:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:11:37.228633 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=31 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:11:37.420986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=25 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:00.070659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f16fd-9d26-4d39-a6af-7277aaa5bbeb None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:00.072233 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0a0f16fd-9d26-4d39-a6af-7277aaa5bbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:00.193515 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f16fd-9d26-4d39-a6af-7277aaa5bbeb None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:00.200074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4216a7a-a9f1-4a13-aa83-9a1102662f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:00.201294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4216a7a-a9f1-4a13-aa83-9a1102662f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:00.201787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4216a7a-a9f1-4a13-aa83-9a1102662f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:00.202278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4216a7a-a9f1-4a13-aa83-9a1102662f3f admin admin] PATH_INFO: `/v3/users/a97447dc4fb643a1b5d8f36e3fc99e38` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:00.204484 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4216a7a-a9f1-4a13-aa83-9a1102662f3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a97447dc4fb643a1b5d8f36e3fc99e38)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:00.208179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4216a7a-a9f1-4a13-aa83-9a1102662f3f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:00.291688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b4216a7a-a9f1-4a13-aa83-9a1102662f3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a97447dc4fb643a1b5d8f36e3fc99e38'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:00.292590 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b4216a7a-a9f1-4a13-aa83-9a1102662f3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a97447dc4fb643a1b5d8f36e3fc99e38'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:00.303310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b4216a7a-a9f1-4a13-aa83-9a1102662f3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a97447dc4fb643a1b5d8f36e3fc99e38'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:00.324202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b4216a7a-a9f1-4a13-aa83-9a1102662f3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a97447dc4fb643a1b5d8f36e3fc99e38'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:00.332825 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b4216a7a-a9f1-4a13-aa83-9a1102662f3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a97447dc4fb643a1b5d8f36e3fc99e38'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:00.340826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 994/1982] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:12:00 2022] DELETE /identity/v3/users/a97447dc4fb643a1b5d8f36e3fc99e38 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:00.347127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b818fd-4029-4cb3-9e24-2a2f54bc465b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:00.347803 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b818fd-4029-4cb3-9e24-2a2f54bc465b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:00.348457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b818fd-4029-4cb3-9e24-2a2f54bc465b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:00.383680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-84b818fd-4029-4cb3-9e24-2a2f54bc465b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:00.386567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-84b818fd-4029-4cb3-9e24-2a2f54bc465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:00.520401 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 989/1983] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:00.531498 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4216a7a-a9f1-4a13-aa83-9a1102662f3f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:00.533203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b4216a7a-a9f1-4a13-aa83-9a1102662f3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:00.641518 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4216a7a-a9f1-4a13-aa83-9a1102662f3f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:00.647434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39603a5-0773-4e45-b8c9-c25e03dfc14e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:00.648451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39603a5-0773-4e45-b8c9-c25e03dfc14e service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:00.648740 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39603a5-0773-4e45-b8c9-c25e03dfc14e service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:00.649134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39603a5-0773-4e45-b8c9-c25e03dfc14e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:00.662351 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39603a5-0773-4e45-b8c9-c25e03dfc14e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:00.664410 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39603a5-0773-4e45-b8c9-c25e03dfc14e service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:00.676409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 995/1984] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:12:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:12:01.204282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b818fd-4029-4cb3-9e24-2a2f54bc465b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:01.206498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-84b818fd-4029-4cb3-9e24-2a2f54bc465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:01.231953 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b818fd-4029-4cb3-9e24-2a2f54bc465b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:01.239786 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa736d-a0bb-489f-b70c-29061b783aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:01.243119 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa736d-a0bb-489f-b70c-29061b783aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:01.243644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa736d-a0bb-489f-b70c-29061b783aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:01.244193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa736d-a0bb-489f-b70c-29061b783aeb admin admin] PATH_INFO: `/v3/projects/725ea091bc4d473eb5b99bfafdf152f6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:01.246467 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa736d-a0bb-489f-b70c-29061b783aeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=725ea091bc4d473eb5b99bfafdf152f6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:01.250396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa736d-a0bb-489f-b70c-29061b783aeb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:01.330509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f8aa736d-a0bb-489f-b70c-29061b783aeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '725ea091bc4d473eb5b99bfafdf152f6', 'request_id': 'req-f8aa736d-a0bb-489f-b70c-29061b783aeb'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:01.339479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f8aa736d-a0bb-489f-b70c-29061b783aeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '725ea091bc4d473eb5b99bfafdf152f6', 'request_id': 'req-f8aa736d-a0bb-489f-b70c-29061b783aeb'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:01.349983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f8aa736d-a0bb-489f-b70c-29061b783aeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '725ea091bc4d473eb5b99bfafdf152f6', 'request_id': 'req-f8aa736d-a0bb-489f-b70c-29061b783aeb'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:01.364793 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f8aa736d-a0bb-489f-b70c-29061b783aeb admin admin] The token cache is being invalidate because project 725ea091bc4d473eb5b99bfafdf152f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:12:01.365144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f8aa736d-a0bb-489f-b70c-29061b783aeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:01.365870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f8aa736d-a0bb-489f-b70c-29061b783aeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:01.368186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 990/1985] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:12:01 2022] DELETE /identity/v3/projects/725ea091bc4d473eb5b99bfafdf152f6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:12:01.374483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5051544-c369-4bb0-9626-bda345c8b1e9 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:01.375031 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5051544-c369-4bb0-9626-bda345c8b1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:01.375151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5051544-c369-4bb0-9626-bda345c8b1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:01.398620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c5051544-c369-4bb0-9626-bda345c8b1e9 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:01.399866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c5051544-c369-4bb0-9626-bda345c8b1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:01.420672 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:01.475144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 996/1986] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:01.484146 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3323ba6-e28e-4ea5-b846-b0d0e284cd53 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:01.484440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3323ba6-e28e-4ea5-b846-b0d0e284cd53 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:01.484727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3323ba6-e28e-4ea5-b846-b0d0e284cd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:01.507482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d3323ba6-e28e-4ea5-b846-b0d0e284cd53 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:01.508880 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d3323ba6-e28e-4ea5-b846-b0d0e284cd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:01.521370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 991/1987] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:01.527437 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5051544-c369-4bb0-9626-bda345c8b1e9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:01.540113 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5051544-c369-4bb0-9626-bda345c8b1e9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:01.545127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2e2fd-812c-49c8-ae43-23362055caf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:01.545818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2e2fd-812c-49c8-ae43-23362055caf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:01.546103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2e2fd-812c-49c8-ae43-23362055caf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:01.546381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2e2fd-812c-49c8-ae43-23362055caf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:01.547130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2e2fd-812c-49c8-ae43-23362055caf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:12:01.547447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2e2fd-812c-49c8-ae43-23362055caf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:01.549186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2e2fd-812c-49c8-ae43-23362055caf2 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:01.566365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 997/1988] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:12:01.571949 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3323ba6-e28e-4ea5-b846-b0d0e284cd53 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:01.582455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3323ba6-e28e-4ea5-b846-b0d0e284cd53 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:01.588798 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6ff71-dcc4-4423-96d6-cd15053ad7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:01.589530 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6ff71-dcc4-4423-96d6-cd15053ad7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:01.589857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6ff71-dcc4-4423-96d6-cd15053ad7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:01.590213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6ff71-dcc4-4423-96d6-cd15053ad7b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:01.590982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6ff71-dcc4-4423-96d6-cd15053ad7b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:01.593163 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6ff71-dcc4-4423-96d6-cd15053ad7b6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:01.630988 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 992/1989] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:12:01 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:12:01.633548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2e2fd-812c-49c8-ae43-23362055caf2 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:01.653456 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2e2fd-812c-49c8-ae43-23362055caf2 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:01.756524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7759791-318e-4b0c-a299-7435b5a06017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:01.757726 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759791-318e-4b0c-a299-7435b5a06017 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:01.758221 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759791-318e-4b0c-a299-7435b5a06017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:01.758718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7759791-318e-4b0c-a299-7435b5a06017 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:01.760069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7759791-318e-4b0c-a299-7435b5a06017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:01.764172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7759791-318e-4b0c-a299-7435b5a06017 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:01.799279 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 998/1990] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:12:01 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:12:01.806199 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6ff71-dcc4-4423-96d6-cd15053ad7b6 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:01.822495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6ff71-dcc4-4423-96d6-cd15053ad7b6 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:01.829668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c65d4c-f9b2-4d05-b38b-c1d7113cfc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:01.830554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c65d4c-f9b2-4d05-b38b-c1d7113cfc0b admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:01.831096 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c65d4c-f9b2-4d05-b38b-c1d7113cfc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:01.831553 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c65d4c-f9b2-4d05-b38b-c1d7113cfc0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:01.832573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c65d4c-f9b2-4d05-b38b-c1d7113cfc0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:01.834547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c65d4c-f9b2-4d05-b38b-c1d7113cfc0b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:01.855603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 993/1991] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:12:01.858518 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7759791-318e-4b0c-a299-7435b5a06017 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:01.871693 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7759791-318e-4b0c-a299-7435b5a06017 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:01.877534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa9933-4442-45bd-9b27-0e51a9e1f490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:01.878237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa9933-4442-45bd-9b27-0e51a9e1f490 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:01.879034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa9933-4442-45bd-9b27-0e51a9e1f490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:01.879034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa9933-4442-45bd-9b27-0e51a9e1f490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:01.879570 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa9933-4442-45bd-9b27-0e51a9e1f490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:01.881767 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa9933-4442-45bd-9b27-0e51a9e1f490 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:01.899883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 999/1992] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:12:01.904617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c65d4c-f9b2-4d05-b38b-c1d7113cfc0b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:01.919273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c65d4c-f9b2-4d05-b38b-c1d7113cfc0b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:01.924993 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8bf2e-da60-4159-9cb8-b255b335edd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:01.926272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8bf2e-da60-4159-9cb8-b255b335edd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:01.926611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8bf2e-da60-4159-9cb8-b255b335edd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:01.926948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8bf2e-da60-4159-9cb8-b255b335edd1 admin admin] PATH_INFO: `/v3/projects/fc771ddd70c74ae2a5f4a7450b5146b7/users/618c09cc43d440b28674e771e4838fff/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:01.949071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8bf2e-da60-4159-9cb8-b255b335edd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc771ddd70c74ae2a5f4a7450b5146b7, user_id=618c09cc43d440b28674e771e4838fff, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:01.954769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8bf2e-da60-4159-9cb8-b255b335edd1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:01.975392 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 994/1993] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:12:01 2022] PUT /identity/v3/projects/fc771ddd70c74ae2a5f4a7450b5146b7/users/618c09cc43d440b28674e771e4838fff/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:01.988822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f4b15-aa19-4a33-85ed-9105a63e645b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:01.989121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f4b15-aa19-4a33-85ed-9105a63e645b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:01.989417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f4b15-aa19-4a33-85ed-9105a63e645b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:02.021579 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e43f4b15-aa19-4a33-85ed-9105a63e645b None None] MFA Rules not processed for user `618c09cc43d440b28674e771e4838fff`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:02.023449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e43f4b15-aa19-4a33-85ed-9105a63e645b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:02.111393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1000/1994] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:02.127188 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8bf2e-da60-4159-9cb8-b255b335edd1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:02.130070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c7a8bf2e-da60-4159-9cb8-b255b335edd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:02.166109 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:02.222375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8bf2e-da60-4159-9cb8-b255b335edd1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:02.228779 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340e50d-2a66-4492-99ae-3ccc85fb3c5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:02.229461 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340e50d-2a66-4492-99ae-3ccc85fb3c5b service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:02.229749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340e50d-2a66-4492-99ae-3ccc85fb3c5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:02.230035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340e50d-2a66-4492-99ae-3ccc85fb3c5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:02.240120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340e50d-2a66-4492-99ae-3ccc85fb3c5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:02.242181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340e50d-2a66-4492-99ae-3ccc85fb3c5b service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:02.253283 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 995/1995] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:12:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:12:11.340907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f4b15-aa19-4a33-85ed-9105a63e645b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:11.342888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e43f4b15-aa19-4a33-85ed-9105a63e645b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:11.377602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:11.457807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f4b15-aa19-4a33-85ed-9105a63e645b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:11.466242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f7b9d-9ceb-4617-801d-14debeb72b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:11.468209 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f7b9d-9ceb-4617-801d-14debeb72b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:11.468764 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f7b9d-9ceb-4617-801d-14debeb72b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:11.469650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f7b9d-9ceb-4617-801d-14debeb72b75 admin admin] PATH_INFO: `/v3/users/971ee571b6b94939a71dac9844c7e352` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:11.473110 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f7b9d-9ceb-4617-801d-14debeb72b75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=971ee571b6b94939a71dac9844c7e352)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:11.478122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f7b9d-9ceb-4617-801d-14debeb72b75 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:11.540647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b54f7b9d-9ceb-4617-801d-14debeb72b75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '971ee571b6b94939a71dac9844c7e352'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:11.541232 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b54f7b9d-9ceb-4617-801d-14debeb72b75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '971ee571b6b94939a71dac9844c7e352'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:11.549945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b54f7b9d-9ceb-4617-801d-14debeb72b75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '971ee571b6b94939a71dac9844c7e352'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:11.565113 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b54f7b9d-9ceb-4617-801d-14debeb72b75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '971ee571b6b94939a71dac9844c7e352'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:11.574130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b54f7b9d-9ceb-4617-801d-14debeb72b75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '971ee571b6b94939a71dac9844c7e352'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:11.581388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1001/1996] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:12:11 2022] DELETE /identity/v3/users/971ee571b6b94939a71dac9844c7e352 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:11.587517 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efa3d5-0962-467c-9f84-7a3e08aa8cd5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:11.588169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efa3d5-0962-467c-9f84-7a3e08aa8cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:11.588709 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efa3d5-0962-467c-9f84-7a3e08aa8cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:11.612667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-55efa3d5-0962-467c-9f84-7a3e08aa8cd5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:11.614767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-55efa3d5-0962-467c-9f84-7a3e08aa8cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:11.735382 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 996/1997] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:11.747820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f7b9d-9ceb-4617-801d-14debeb72b75 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:11.749451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b54f7b9d-9ceb-4617-801d-14debeb72b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:11.824671 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f7b9d-9ceb-4617-801d-14debeb72b75 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:11.828562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e03af5-954a-483b-b7db-c81663aeabaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:11.829236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03af5-954a-483b-b7db-c81663aeabaf service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:11.829525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03af5-954a-483b-b7db-c81663aeabaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:11.829805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03af5-954a-483b-b7db-c81663aeabaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:11.842370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e03af5-954a-483b-b7db-c81663aeabaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:11.845083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e03af5-954a-483b-b7db-c81663aeabaf service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:11.858596 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1002/1998] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:12:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:12:12.304810 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55efa3d5-0962-467c-9f84-7a3e08aa8cd5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:12.307806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-55efa3d5-0962-467c-9f84-7a3e08aa8cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:12.346942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55efa3d5-0962-467c-9f84-7a3e08aa8cd5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:12.357204 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4c27f-2a41-47fc-a065-cc3b0f7aa113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:12.358718 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4c27f-2a41-47fc-a065-cc3b0f7aa113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:12.359193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4c27f-2a41-47fc-a065-cc3b0f7aa113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:12.359826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4c27f-2a41-47fc-a065-cc3b0f7aa113 admin admin] PATH_INFO: `/v3/projects/d3149de22a83431fb4f39a957010c9b5` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:12.364459 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4c27f-2a41-47fc-a065-cc3b0f7aa113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3149de22a83431fb4f39a957010c9b5)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:12.368053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4c27f-2a41-47fc-a065-cc3b0f7aa113 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:12.440796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-30d4c27f-2a41-47fc-a065-cc3b0f7aa113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3149de22a83431fb4f39a957010c9b5', 'request_id': 'req-30d4c27f-2a41-47fc-a065-cc3b0f7aa113'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:12.458898 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-30d4c27f-2a41-47fc-a065-cc3b0f7aa113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3149de22a83431fb4f39a957010c9b5', 'request_id': 'req-30d4c27f-2a41-47fc-a065-cc3b0f7aa113'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:12.469597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-30d4c27f-2a41-47fc-a065-cc3b0f7aa113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3149de22a83431fb4f39a957010c9b5', 'request_id': 'req-30d4c27f-2a41-47fc-a065-cc3b0f7aa113'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:12.484760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-30d4c27f-2a41-47fc-a065-cc3b0f7aa113 admin admin] The token cache is being invalidate because project d3149de22a83431fb4f39a957010c9b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:12:12.485084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-30d4c27f-2a41-47fc-a065-cc3b0f7aa113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:12.485931 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-30d4c27f-2a41-47fc-a065-cc3b0f7aa113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:12.487601 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 997/1999] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:12:12 2022] DELETE /identity/v3/projects/d3149de22a83431fb4f39a957010c9b5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:12:12.498294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f73159-ff9c-4832-9b4d-3997656203dc None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:12.498923 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f73159-ff9c-4832-9b4d-3997656203dc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:12.498923 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f73159-ff9c-4832-9b4d-3997656203dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:12.520344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f0f73159-ff9c-4832-9b4d-3997656203dc None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:12.521710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f0f73159-ff9c-4832-9b4d-3997656203dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:12.573392 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:12.613494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1003/2000] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:12.623540 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00da0d-153f-4c32-99f8-a6592f2220ff None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:12.624427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00da0d-153f-4c32-99f8-a6592f2220ff None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:12.624583 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00da0d-153f-4c32-99f8-a6592f2220ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:12.645787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4d00da0d-153f-4c32-99f8-a6592f2220ff None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:12.647746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4d00da0d-153f-4c32-99f8-a6592f2220ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:12.656703 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 998/2001] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:12.661389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f73159-ff9c-4832-9b4d-3997656203dc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:12.673140 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f73159-ff9c-4832-9b4d-3997656203dc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:12.679500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261e5c3-e7d9-4759-a30b-43f81e275bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:12.680491 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261e5c3-e7d9-4759-a30b-43f81e275bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:12.680829 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261e5c3-e7d9-4759-a30b-43f81e275bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:12.681066 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261e5c3-e7d9-4759-a30b-43f81e275bc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:12.681841 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261e5c3-e7d9-4759-a30b-43f81e275bc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:12:12.682300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261e5c3-e7d9-4759-a30b-43f81e275bc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:12.684234 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261e5c3-e7d9-4759-a30b-43f81e275bc5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:12.700054 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1004/2002] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 09:12:12.704911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00da0d-153f-4c32-99f8-a6592f2220ff None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:12.716146 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00da0d-153f-4c32-99f8-a6592f2220ff None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:12.722007 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c32e1-6e10-4588-bb27-9391c98cc55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:12.722954 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c32e1-6e10-4588-bb27-9391c98cc55e admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:12.723310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c32e1-6e10-4588-bb27-9391c98cc55e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:12.723597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c32e1-6e10-4588-bb27-9391c98cc55e admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:12.724370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c32e1-6e10-4588-bb27-9391c98cc55e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:12.726972 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c32e1-6e10-4588-bb27-9391c98cc55e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:12.766142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 999/2003] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:12:12 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:12:12.772637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261e5c3-e7d9-4759-a30b-43f81e275bc5 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:12.787451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261e5c3-e7d9-4759-a30b-43f81e275bc5 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:12.918656 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523dc2e-8196-44ad-ba6b-15a2717a9800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:12.919727 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523dc2e-8196-44ad-ba6b-15a2717a9800 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:12.920226 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523dc2e-8196-44ad-ba6b-15a2717a9800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:12.920685 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523dc2e-8196-44ad-ba6b-15a2717a9800 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:12.921931 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523dc2e-8196-44ad-ba6b-15a2717a9800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:12.925749 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523dc2e-8196-44ad-ba6b-15a2717a9800 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:12.958189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1005/2004] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:12:12 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:12:12.964153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c32e1-6e10-4588-bb27-9391c98cc55e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:12.976855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c32e1-6e10-4588-bb27-9391c98cc55e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:12.984341 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e53310-70cc-477a-a675-4db74500fba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:12.985218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e53310-70cc-477a-a675-4db74500fba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:12.985596 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e53310-70cc-477a-a675-4db74500fba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:12.985962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e53310-70cc-477a-a675-4db74500fba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:12.986948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e53310-70cc-477a-a675-4db74500fba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:12.989341 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e53310-70cc-477a-a675-4db74500fba6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:13.004718 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1000/2005] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:12:13.015556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523dc2e-8196-44ad-ba6b-15a2717a9800 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:13.029873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523dc2e-8196-44ad-ba6b-15a2717a9800 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:13.038146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb799160-4233-4c88-b716-1bea9ba1e617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:13.039249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb799160-4233-4c88-b716-1bea9ba1e617 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:13.039712 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb799160-4233-4c88-b716-1bea9ba1e617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:13.040205 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb799160-4233-4c88-b716-1bea9ba1e617 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:13.041489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb799160-4233-4c88-b716-1bea9ba1e617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:13.050564 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb799160-4233-4c88-b716-1bea9ba1e617 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:13.078928 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1006/2006] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:12:13.080642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e53310-70cc-477a-a675-4db74500fba6 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:13.101660 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e53310-70cc-477a-a675-4db74500fba6 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:13.109474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808db3cc-7ef0-47f9-801c-ab909c7cfe16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:13.111012 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808db3cc-7ef0-47f9-801c-ab909c7cfe16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:13.111575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808db3cc-7ef0-47f9-801c-ab909c7cfe16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:13.112057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808db3cc-7ef0-47f9-801c-ab909c7cfe16 admin admin] PATH_INFO: `/v3/projects/eb15baa22fc949f49f215d65dab0f3b1/users/5445e009ab664626a2ab54351b0a9631/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:13.117068 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:13.145583 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808db3cc-7ef0-47f9-801c-ab909c7cfe16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb15baa22fc949f49f215d65dab0f3b1, user_id=5445e009ab664626a2ab54351b0a9631, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:13.150307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808db3cc-7ef0-47f9-801c-ab909c7cfe16 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:13.180309 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1001/2007] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:12:13 2022] PUT /identity/v3/projects/eb15baa22fc949f49f215d65dab0f3b1/users/5445e009ab664626a2ab54351b0a9631/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:13.187411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6484e-65e0-41eb-97be-b8e3a583d340 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:13.187677 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6484e-65e0-41eb-97be-b8e3a583d340 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:13.188226 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6484e-65e0-41eb-97be-b8e3a583d340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:13.226068 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-74f6484e-65e0-41eb-97be-b8e3a583d340 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:13.227973 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-74f6484e-65e0-41eb-97be-b8e3a583d340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:13.308587 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1007/2008] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:13.320772 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808db3cc-7ef0-47f9-801c-ab909c7cfe16 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:13.323149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-808db3cc-7ef0-47f9-801c-ab909c7cfe16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:13.401740 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808db3cc-7ef0-47f9-801c-ab909c7cfe16 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:13.406260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec787b5-2f56-4e9c-9e8c-dd212a8dfd92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:13.407191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec787b5-2f56-4e9c-9e8c-dd212a8dfd92 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:13.407661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec787b5-2f56-4e9c-9e8c-dd212a8dfd92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:13.408120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec787b5-2f56-4e9c-9e8c-dd212a8dfd92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:13.421514 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec787b5-2f56-4e9c-9e8c-dd212a8dfd92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:13.423064 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec787b5-2f56-4e9c-9e8c-dd212a8dfd92 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:13.438251 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1002/2009] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:12:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:12:22.784396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6484e-65e0-41eb-97be-b8e3a583d340 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:22.786852 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-74f6484e-65e0-41eb-97be-b8e3a583d340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:22.803025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec787b5-2f56-4e9c-9e8c-dd212a8dfd92 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:22.805595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8ec787b5-2f56-4e9c-9e8c-dd212a8dfd92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:22.839071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec787b5-2f56-4e9c-9e8c-dd212a8dfd92 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:22.847531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195056d-135e-45ae-a8d7-6b039de8bbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:22.848770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195056d-135e-45ae-a8d7-6b039de8bbd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:22.849315 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195056d-135e-45ae-a8d7-6b039de8bbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:22.850543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195056d-135e-45ae-a8d7-6b039de8bbd5 admin admin] PATH_INFO: `/v3/users/675378ce4e594727a59049f6ce476ce0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:22.852279 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195056d-135e-45ae-a8d7-6b039de8bbd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=675378ce4e594727a59049f6ce476ce0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:22.852803 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6484e-65e0-41eb-97be-b8e3a583d340 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:22.857121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195056d-135e-45ae-a8d7-6b039de8bbd5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:22.874779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f17526-df6a-4d99-bcca-3076cf7b824b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:22.876421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f17526-df6a-4d99-bcca-3076cf7b824b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:22.877009 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f17526-df6a-4d99-bcca-3076cf7b824b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:22.877429 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f17526-df6a-4d99-bcca-3076cf7b824b admin admin] PATH_INFO: `/v3/users/618c09cc43d440b28674e771e4838fff` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:22.879949 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f17526-df6a-4d99-bcca-3076cf7b824b admin admin] RBAC: Authorizing `identity:delete_user(user_id=618c09cc43d440b28674e771e4838fff)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:22.885290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f17526-df6a-4d99-bcca-3076cf7b824b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:22.967727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f195056d-135e-45ae-a8d7-6b039de8bbd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '675378ce4e594727a59049f6ce476ce0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:22.985904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f195056d-135e-45ae-a8d7-6b039de8bbd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '675378ce4e594727a59049f6ce476ce0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:23.003322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f195056d-135e-45ae-a8d7-6b039de8bbd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '675378ce4e594727a59049f6ce476ce0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:23.023303 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:23.036915 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65f17526-df6a-4d99-bcca-3076cf7b824b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '618c09cc43d440b28674e771e4838fff'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:23.039679 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65f17526-df6a-4d99-bcca-3076cf7b824b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '618c09cc43d440b28674e771e4838fff'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:23.059634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f195056d-135e-45ae-a8d7-6b039de8bbd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '675378ce4e594727a59049f6ce476ce0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:23.060684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f195056d-135e-45ae-a8d7-6b039de8bbd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '675378ce4e594727a59049f6ce476ce0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:23.065227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65f17526-df6a-4d99-bcca-3076cf7b824b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '618c09cc43d440b28674e771e4838fff'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:23.080152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1003/2010] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:12:22 2022] DELETE /identity/v3/users/675378ce4e594727a59049f6ce476ce0 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:23.116843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65f17526-df6a-4d99-bcca-3076cf7b824b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '618c09cc43d440b28674e771e4838fff'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:23.145368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65f17526-df6a-4d99-bcca-3076cf7b824b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '618c09cc43d440b28674e771e4838fff'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:23.163483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1008/2011] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:12:22 2022] DELETE /identity/v3/users/618c09cc43d440b28674e771e4838fff => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:23.166100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a0f60-a6a1-4ba1-9788-6b60ba19393c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:23.167635 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a0f60-a6a1-4ba1-9788-6b60ba19393c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:23.168752 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a0f60-a6a1-4ba1-9788-6b60ba19393c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:23.243465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-650a0f60-a6a1-4ba1-9788-6b60ba19393c None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:23.245207 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-650a0f60-a6a1-4ba1-9788-6b60ba19393c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:23.441767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:23.453468 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f17526-df6a-4d99-bcca-3076cf7b824b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:23.458378 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-65f17526-df6a-4d99-bcca-3076cf7b824b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:23.480224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1004/2012] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:12:23.497458 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f17526-df6a-4d99-bcca-3076cf7b824b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:23.514824 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2fba5-7753-4061-869f-d842ab27dabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:23.517382 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2fba5-7753-4061-869f-d842ab27dabe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:23.518134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2fba5-7753-4061-869f-d842ab27dabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:23.518723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2fba5-7753-4061-869f-d842ab27dabe admin admin] PATH_INFO: `/v3/projects/808a7ff7e5f848f6aa646b09641551c4` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:23.522345 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2fba5-7753-4061-869f-d842ab27dabe admin admin] RBAC: Authorizing `identity:delete_project(project_id=808a7ff7e5f848f6aa646b09641551c4)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:23.533265 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2fba5-7753-4061-869f-d842ab27dabe admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:23.536802 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a0f60-a6a1-4ba1-9788-6b60ba19393c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:23.539019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-650a0f60-a6a1-4ba1-9788-6b60ba19393c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:23.677536 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a0f60-a6a1-4ba1-9788-6b60ba19393c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:23.703359 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65f2fba5-7753-4061-869f-d842ab27dabe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '808a7ff7e5f848f6aa646b09641551c4', 'request_id': 'req-65f2fba5-7753-4061-869f-d842ab27dabe'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:23.724306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65f2fba5-7753-4061-869f-d842ab27dabe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '808a7ff7e5f848f6aa646b09641551c4', 'request_id': 'req-65f2fba5-7753-4061-869f-d842ab27dabe'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:23.732794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65f2fba5-7753-4061-869f-d842ab27dabe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '808a7ff7e5f848f6aa646b09641551c4', 'request_id': 'req-65f2fba5-7753-4061-869f-d842ab27dabe'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:23.753389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65f2fba5-7753-4061-869f-d842ab27dabe admin admin] The token cache is being invalidate because project 808a7ff7e5f848f6aa646b09641551c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:12:23.753688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65f2fba5-7753-4061-869f-d842ab27dabe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:23.754407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65f2fba5-7753-4061-869f-d842ab27dabe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:23.760699 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1009/2013] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:12:23 2022] DELETE /identity/v3/projects/808a7ff7e5f848f6aa646b09641551c4 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:23.762470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0205bb-d9f2-417d-a5a7-b8732cb98cc5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:23.762766 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0205bb-d9f2-417d-a5a7-b8732cb98cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:23.763052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0205bb-d9f2-417d-a5a7-b8732cb98cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:23.801569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6f0205bb-d9f2-417d-a5a7-b8732cb98cc5 None None] MFA Rules not processed for user `5445e009ab664626a2ab54351b0a9631`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:23.803852 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6f0205bb-d9f2-417d-a5a7-b8732cb98cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:23.817010 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16eaaa-132d-4b6e-aad8-c03740e2e52a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:23.822102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16eaaa-132d-4b6e-aad8-c03740e2e52a service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:23.822102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16eaaa-132d-4b6e-aad8-c03740e2e52a service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:23.823085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16eaaa-132d-4b6e-aad8-c03740e2e52a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:23.828088 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dc16eaaa-132d-4b6e-aad8-c03740e2e52a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:23.920158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1010/2014] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:23.921590 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482b669-c518-4b47-a2f3-08350637180c None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:23.921885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482b669-c518-4b47-a2f3-08350637180c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:23.922185 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482b669-c518-4b47-a2f3-08350637180c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:23.928641 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16eaaa-132d-4b6e-aad8-c03740e2e52a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:23.930773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16eaaa-132d-4b6e-aad8-c03740e2e52a service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:23.945633 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c482b669-c518-4b47-a2f3-08350637180c None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:23.947004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c482b669-c518-4b47-a2f3-08350637180c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:23.948874 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1005/2015] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:12:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:12:23.956505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1011/2016] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:23.963273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16eaaa-132d-4b6e-aad8-c03740e2e52a service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:23.965181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7c298-3e82-4646-9b0e-542861c0dd6b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:23.965696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7c298-3e82-4646-9b0e-542861c0dd6b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:23.965935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7c298-3e82-4646-9b0e-542861c0dd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:23.966740 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dc16eaaa-132d-4b6e-aad8-c03740e2e52a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:23.987184 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16eaaa-132d-4b6e-aad8-c03740e2e52a service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:23.995353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-99b7c298-3e82-4646-9b0e-542861c0dd6b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:23.996683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-99b7c298-3e82-4646-9b0e-542861c0dd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:23.999607 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd276634-daea-4c11-ac2b-ff7c134960b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:24.000413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd276634-daea-4c11-ac2b-ff7c134960b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:24.000803 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd276634-daea-4c11-ac2b-ff7c134960b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:24.001079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd276634-daea-4c11-ac2b-ff7c134960b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:24.001801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd276634-daea-4c11-ac2b-ff7c134960b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:24.010825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd276634-daea-4c11-ac2b-ff7c134960b4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:24.016854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1012/2017] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:12:24.018878 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7c298-3e82-4646-9b0e-542861c0dd6b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:24.038934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b7c298-3e82-4646-9b0e-542861c0dd6b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:24.053339 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcbba6-73c9-4aaa-899d-f12c8a708d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:24.054384 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcbba6-73c9-4aaa-899d-f12c8a708d7b admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:24.054549 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcbba6-73c9-4aaa-899d-f12c8a708d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:24.054877 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcbba6-73c9-4aaa-899d-f12c8a708d7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:24.055600 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fcbba6-73c9-4aaa-899d-f12c8a708d7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:12:24.055962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fcbba6-73c9-4aaa-899d-f12c8a708d7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:24.057809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fcbba6-73c9-4aaa-899d-f12c8a708d7b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:24.068083 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1006/2018] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:12:23 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:12:24.072549 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd276634-daea-4c11-ac2b-ff7c134960b4 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:24.096301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1013/2019] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:12:24.098598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcbba6-73c9-4aaa-899d-f12c8a708d7b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:24.117943 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:24.127977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcbba6-73c9-4aaa-899d-f12c8a708d7b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:24.221160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:24.268266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ef304-962d-4d23-9a4c-e2cab94f420d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:24.273949 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ef304-962d-4d23-9a4c-e2cab94f420d admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:24.274311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ef304-962d-4d23-9a4c-e2cab94f420d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:24.274615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ef304-962d-4d23-9a4c-e2cab94f420d admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:24.275965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ef304-962d-4d23-9a4c-e2cab94f420d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:24.282109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ef304-962d-4d23-9a4c-e2cab94f420d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:24.305447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd276634-daea-4c11-ac2b-ff7c134960b4 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:24.328815 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f308b49-8ae4-4935-a767-34f7bdf51487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:24.334931 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1014/2020] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:12:24 2022] POST /identity/v3/projects => generated 405 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:12:24.335193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f308b49-8ae4-4935-a767-34f7bdf51487 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:24.335193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f308b49-8ae4-4935-a767-34f7bdf51487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:24.335193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f308b49-8ae4-4935-a767-34f7bdf51487 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:24.335193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f308b49-8ae4-4935-a767-34f7bdf51487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:24.336342 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ef304-962d-4d23-9a4c-e2cab94f420d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:24.342003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f308b49-8ae4-4935-a767-34f7bdf51487 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:24.431396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1007/2021] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:12:24 2022] POST /identity/v3/users => generated 295 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:12:24.433272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f308b49-8ae4-4935-a767-34f7bdf51487 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:24.478414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f308b49-8ae4-4935-a767-34f7bdf51487 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:24.538803 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ef304-962d-4d23-9a4c-e2cab94f420d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:24.560385 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd2e43-4688-4579-b2d2-030d3255114c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:24.563058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd2e43-4688-4579-b2d2-030d3255114c admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:24.563642 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd2e43-4688-4579-b2d2-030d3255114c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:24.565034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd2e43-4688-4579-b2d2-030d3255114c admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:24.571455 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd2e43-4688-4579-b2d2-030d3255114c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:24.575045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd2e43-4688-4579-b2d2-030d3255114c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:24.621441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1015/2022] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:12:24 2022] POST /identity/v3/users => generated 303 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:12:24.627358 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd2e43-4688-4579-b2d2-030d3255114c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:24.682936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd2e43-4688-4579-b2d2-030d3255114c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:24.692725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d8838-281d-42d7-81f6-5a7d1cfe812d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:24.692976 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915e432-887f-49aa-9b09-eb3336789f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:24.694589 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d8838-281d-42d7-81f6-5a7d1cfe812d admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:24.694983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915e432-887f-49aa-9b09-eb3336789f20 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:24.695188 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d8838-281d-42d7-81f6-5a7d1cfe812d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:24.695683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d8838-281d-42d7-81f6-5a7d1cfe812d admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:24.696384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915e432-887f-49aa-9b09-eb3336789f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:24.696974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915e432-887f-49aa-9b09-eb3336789f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:24.697288 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d8838-281d-42d7-81f6-5a7d1cfe812d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:24.698545 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915e432-887f-49aa-9b09-eb3336789f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:24.700707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d8838-281d-42d7-81f6-5a7d1cfe812d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:24.702413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915e432-887f-49aa-9b09-eb3336789f20 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:24.741678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1016/2023] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:12:24.742992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d8838-281d-42d7-81f6-5a7d1cfe812d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:24.745750 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1008/2024] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:12:24.748137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915e432-887f-49aa-9b09-eb3336789f20 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:24.754703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-997d8838-281d-42d7-81f6-5a7d1cfe812d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:24.769335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915e432-887f-49aa-9b09-eb3336789f20 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:24.780242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909e7e7-17c0-4d62-bfb7-88955997cce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:24.781551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909e7e7-17c0-4d62-bfb7-88955997cce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:24.782153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909e7e7-17c0-4d62-bfb7-88955997cce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:24.782729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909e7e7-17c0-4d62-bfb7-88955997cce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:24.784247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909e7e7-17c0-4d62-bfb7-88955997cce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:24.786156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d8838-281d-42d7-81f6-5a7d1cfe812d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:24.790654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909e7e7-17c0-4d62-bfb7-88955997cce2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:24.796793 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:24.798779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:24.799504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:24.800706 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89 admin admin] PATH_INFO: `/v3/projects/fc771ddd70c74ae2a5f4a7450b5146b7` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:24.805836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc771ddd70c74ae2a5f4a7450b5146b7)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:24.809360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:24.812796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1009/2025] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:12:24.814195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909e7e7-17c0-4d62-bfb7-88955997cce2 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:24.838955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909e7e7-17c0-4d62-bfb7-88955997cce2 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:24.846890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09219b87-75d4-4dd3-974c-c1dcd595e77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:24.848836 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09219b87-75d4-4dd3-974c-c1dcd595e77f admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:24.849560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09219b87-75d4-4dd3-974c-c1dcd595e77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:24.850589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09219b87-75d4-4dd3-974c-c1dcd595e77f admin admin] PATH_INFO: `/v3/projects/c60bdadef20a4b61b0f6f8db1343d630/users/83a97e1d803043acadb43266a0bc0cd6/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:24.880567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09219b87-75d4-4dd3-974c-c1dcd595e77f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60bdadef20a4b61b0f6f8db1343d630, user_id=83a97e1d803043acadb43266a0bc0cd6, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:24.887306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09219b87-75d4-4dd3-974c-c1dcd595e77f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:24.908438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc771ddd70c74ae2a5f4a7450b5146b7', 'request_id': 'req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:24.920706 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1010/2026] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:12:24 2022] PUT /identity/v3/projects/c60bdadef20a4b61b0f6f8db1343d630/users/83a97e1d803043acadb43266a0bc0cd6/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:24.921185 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc771ddd70c74ae2a5f4a7450b5146b7', 'request_id': 'req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:24.922333 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09219b87-75d4-4dd3-974c-c1dcd595e77f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:24.937216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc771ddd70c74ae2a5f4a7450b5146b7', 'request_id': 'req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:24.940955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:24.979159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89 admin admin] The token cache is being invalidate because project fc771ddd70c74ae2a5f4a7450b5146b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:12:24.979770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:24.981715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:24.985233 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1017/2027] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:12:24 2022] DELETE /identity/v3/projects/fc771ddd70c74ae2a5f4a7450b5146b7 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:24.987257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:24.993312 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:25.098932 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09219b87-75d4-4dd3-974c-c1dcd595e77f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:25.106765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eafe34-2186-4e92-90c7-e9ca453c5d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:25.108458 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eafe34-2186-4e92-90c7-e9ca453c5d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:25.108843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eafe34-2186-4e92-90c7-e9ca453c5d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:25.109511 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eafe34-2186-4e92-90c7-e9ca453c5d15 admin admin] PATH_INFO: `/v3/projects/2cbd7fa520874db497a00b3ab41acee6/users/13439b8b7a5b4a3f9038fcecadc2d03f/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:25.141745 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eafe34-2186-4e92-90c7-e9ca453c5d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cbd7fa520874db497a00b3ab41acee6, user_id=13439b8b7a5b4a3f9038fcecadc2d03f, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:25.145139 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eafe34-2186-4e92-90c7-e9ca453c5d15 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:25.173477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61c388-b4d2-4aeb-a6f8-f8f0305d2b89 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:25.182143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1011/2028] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:12:24 2022] PUT /identity/v3/projects/2cbd7fa520874db497a00b3ab41acee6/users/13439b8b7a5b4a3f9038fcecadc2d03f/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:25.182143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a7ba0-975d-4723-9778-a7a96c0fb706 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:25.182143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a7ba0-975d-4723-9778-a7a96c0fb706 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:25.182497 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a7ba0-975d-4723-9778-a7a96c0fb706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:25.231371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-798a7ba0-975d-4723-9778-a7a96c0fb706 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:25.231371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-798a7ba0-975d-4723-9778-a7a96c0fb706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:25.247848 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:25.270278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5da115-873e-43de-b2e6-74e3b4c840cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:25.271095 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5da115-873e-43de-b2e6-74e3b4c840cf admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:25.273281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5da115-873e-43de-b2e6-74e3b4c840cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:25.273711 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5da115-873e-43de-b2e6-74e3b4c840cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:25.274586 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5da115-873e-43de-b2e6-74e3b4c840cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:25.276832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5da115-873e-43de-b2e6-74e3b4c840cf admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:25.299454 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1018/2029] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:12:25.301963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb737f-6913-4e04-b1a8-7343e7505432 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:25.302438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb737f-6913-4e04-b1a8-7343e7505432 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:25.303505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb737f-6913-4e04-b1a8-7343e7505432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:25.330499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-89eb737f-6913-4e04-b1a8-7343e7505432 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:25.332099 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-89eb737f-6913-4e04-b1a8-7343e7505432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:25.343139 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1019/2030] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:25.344144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb737f-6913-4e04-b1a8-7343e7505432 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:25.364646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1012/2031] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:25.366957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a7ba0-975d-4723-9778-a7a96c0fb706 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:25.368881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-798a7ba0-975d-4723-9778-a7a96c0fb706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:25.372338 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb737f-6913-4e04-b1a8-7343e7505432 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:25.379070 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2814f-00ef-4981-80cc-1c01460518f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:25.380254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2814f-00ef-4981-80cc-1c01460518f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:25.381531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2814f-00ef-4981-80cc-1c01460518f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:25.381891 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2814f-00ef-4981-80cc-1c01460518f4 admin admin] PATH_INFO: `/v3/projects/c60bdadef20a4b61b0f6f8db1343d630/users/83a97e1d803043acadb43266a0bc0cd6/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:25.385051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2814f-00ef-4981-80cc-1c01460518f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60bdadef20a4b61b0f6f8db1343d630, user_id=83a97e1d803043acadb43266a0bc0cd6, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:25.388202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2814f-00ef-4981-80cc-1c01460518f4 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:25.409547 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1020/2032] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:12:25 2022] PUT /identity/v3/projects/c60bdadef20a4b61b0f6f8db1343d630/users/83a97e1d803043acadb43266a0bc0cd6/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:25.411999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898bbfd-2cf7-4128-bfce-b3773c32e87c None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:25.412557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898bbfd-2cf7-4128-bfce-b3773c32e87c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:25.413459 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898bbfd-2cf7-4128-bfce-b3773c32e87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:25.446343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b898bbfd-2cf7-4128-bfce-b3773c32e87c None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:25.447836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b898bbfd-2cf7-4128-bfce-b3773c32e87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:25.483659 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a7ba0-975d-4723-9778-a7a96c0fb706 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:25.574347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f691c2d-0aee-4a4a-87ca-6c4fdedb9d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:25.574347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f691c2d-0aee-4a4a-87ca-6c4fdedb9d11 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:25.574719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f691c2d-0aee-4a4a-87ca-6c4fdedb9d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:25.575082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f691c2d-0aee-4a4a-87ca-6c4fdedb9d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:25.610536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1021/2033] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:25.614015 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898bbfd-2cf7-4128-bfce-b3773c32e87c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:25.650692 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898bbfd-2cf7-4128-bfce-b3773c32e87c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:25.658702 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8eada-49b5-425c-a6b3-a9174f32c2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:25.661483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8eada-49b5-425c-a6b3-a9174f32c2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:25.663686 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8eada-49b5-425c-a6b3-a9174f32c2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:25.664453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8eada-49b5-425c-a6b3-a9174f32c2c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:25.666671 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8eada-49b5-425c-a6b3-a9174f32c2c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:12:25.667272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8eada-49b5-425c-a6b3-a9174f32c2c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:25.671200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be8eada-49b5-425c-a6b3-a9174f32c2c3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:25.716097 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1022/2034] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:12:25.716097 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8eada-49b5-425c-a6b3-a9174f32c2c3 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:25.716737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f691c2d-0aee-4a4a-87ca-6c4fdedb9d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:25.719991 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f691c2d-0aee-4a4a-87ca-6c4fdedb9d11 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:25.732466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:25.734223 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1013/2035] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:12:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:12:25.748655 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8eada-49b5-425c-a6b3-a9174f32c2c3 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:25.768933 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec918704-a826-486f-87a7-bc375dc4ae9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:25.770576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec918704-a826-486f-87a7-bc375dc4ae9d admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:25.771044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec918704-a826-486f-87a7-bc375dc4ae9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:25.772164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec918704-a826-486f-87a7-bc375dc4ae9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:25.773529 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec918704-a826-486f-87a7-bc375dc4ae9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:25.776261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec918704-a826-486f-87a7-bc375dc4ae9d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:25.823110 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1023/2036] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:12:25 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:12:25.830328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f691c2d-0aee-4a4a-87ca-6c4fdedb9d11 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:25.844161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f691c2d-0aee-4a4a-87ca-6c4fdedb9d11 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:25.884500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:26.002272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc793a5d-a3ac-4f0f-959f-608b30bcb68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:26.003243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc793a5d-a3ac-4f0f-959f-608b30bcb68f admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:26.003578 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc793a5d-a3ac-4f0f-959f-608b30bcb68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:26.003815 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc793a5d-a3ac-4f0f-959f-608b30bcb68f admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:26.004842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc793a5d-a3ac-4f0f-959f-608b30bcb68f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:26.027093 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc793a5d-a3ac-4f0f-959f-608b30bcb68f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:26.077202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1014/2037] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:12:25 2022] POST /identity/v3/users => generated 292 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:12:26.083884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec918704-a826-486f-87a7-bc375dc4ae9d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:26.103996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec918704-a826-486f-87a7-bc375dc4ae9d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:26.119190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd8cc8-ede2-4151-9328-ad9d8da00cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:26.119771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd8cc8-ede2-4151-9328-ad9d8da00cbe admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:26.120107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd8cc8-ede2-4151-9328-ad9d8da00cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:26.120386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd8cc8-ede2-4151-9328-ad9d8da00cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:26.121411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd8cc8-ede2-4151-9328-ad9d8da00cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:26.123194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd8cc8-ede2-4151-9328-ad9d8da00cbe admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:26.147380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1024/2038] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:12:26.154620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc793a5d-a3ac-4f0f-959f-608b30bcb68f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:26.177781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc793a5d-a3ac-4f0f-959f-608b30bcb68f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:26.191067 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ba79a-caa2-4934-97cd-a1f61aa1e581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:26.191854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ba79a-caa2-4934-97cd-a1f61aa1e581 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:26.192273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ba79a-caa2-4934-97cd-a1f61aa1e581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:26.192683 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ba79a-caa2-4934-97cd-a1f61aa1e581 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:26.194763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ba79a-caa2-4934-97cd-a1f61aa1e581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:26.196806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ba79a-caa2-4934-97cd-a1f61aa1e581 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:26.216701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1015/2039] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:12:26.218014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd8cc8-ede2-4151-9328-ad9d8da00cbe admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:26.235131 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd8cc8-ede2-4151-9328-ad9d8da00cbe admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:26.251348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e685014-85d4-4369-84bf-22d0381bf952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:26.252297 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e685014-85d4-4369-84bf-22d0381bf952 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:26.252587 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e685014-85d4-4369-84bf-22d0381bf952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:26.252870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e685014-85d4-4369-84bf-22d0381bf952 admin admin] PATH_INFO: `/v3/projects/afdbd56ebd1c48f1aa35d37819e000fa/users/326bf42fe8c642b4898f2c8380733599/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:26.290460 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e685014-85d4-4369-84bf-22d0381bf952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afdbd56ebd1c48f1aa35d37819e000fa, user_id=326bf42fe8c642b4898f2c8380733599, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:26.298472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e685014-85d4-4369-84bf-22d0381bf952 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:26.327990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1025/2040] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:12:26 2022] PUT /identity/v3/projects/afdbd56ebd1c48f1aa35d37819e000fa/users/326bf42fe8c642b4898f2c8380733599/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:12:26.333056 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bb3db-d1f1-4c44-8e52-b76471eaed66 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:26.334019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bb3db-d1f1-4c44-8e52-b76471eaed66 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:26.334645 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bb3db-d1f1-4c44-8e52-b76471eaed66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:26.366655 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-861bb3db-d1f1-4c44-8e52-b76471eaed66 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:26.369189 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-861bb3db-d1f1-4c44-8e52-b76471eaed66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:26.541447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1016/2041] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:12:26.554924 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e685014-85d4-4369-84bf-22d0381bf952 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:26.557788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8e685014-85d4-4369-84bf-22d0381bf952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:26.695425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e685014-85d4-4369-84bf-22d0381bf952 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:26.703166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9526325-137a-4f08-9900-dc43731cde68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:26.704650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9526325-137a-4f08-9900-dc43731cde68 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:26.704650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9526325-137a-4f08-9900-dc43731cde68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:26.705748 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9526325-137a-4f08-9900-dc43731cde68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:26.724303 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9526325-137a-4f08-9900-dc43731cde68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:26.726531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9526325-137a-4f08-9900-dc43731cde68 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:26.770887 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1026/2042] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:12:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:12:27.162810 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acc19d-dae2-4106-bc55-0f69ca33bb76 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:27.164435 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acc19d-dae2-4106-bc55-0f69ca33bb76 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:27.165011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32acc19d-dae2-4106-bc55-0f69ca33bb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:27.242943 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-32acc19d-dae2-4106-bc55-0f69ca33bb76 None None] MFA Rules not processed for user `13439b8b7a5b4a3f9038fcecadc2d03f`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:27.244370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-32acc19d-dae2-4106-bc55-0f69ca33bb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:27.356672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1017/2043] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:27 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:27.361084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9526325-137a-4f08-9900-dc43731cde68 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:27.362762 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d9526325-137a-4f08-9900-dc43731cde68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:27.394843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9526325-137a-4f08-9900-dc43731cde68 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:27.411493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900176a-c3ce-405c-9d9a-7c259a2cca58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:27.413051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900176a-c3ce-405c-9d9a-7c259a2cca58 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:27.413536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900176a-c3ce-405c-9d9a-7c259a2cca58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:27.413961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900176a-c3ce-405c-9d9a-7c259a2cca58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:27.414847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900176a-c3ce-405c-9d9a-7c259a2cca58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:27.417037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900176a-c3ce-405c-9d9a-7c259a2cca58 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:27.454341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1027/2044] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:12:27 2022] POST /identity/v3/projects => generated 401 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:12:27.456018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acc19d-dae2-4106-bc55-0f69ca33bb76 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:27.519264 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32acc19d-dae2-4106-bc55-0f69ca33bb76 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:27.596171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:27.720048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519d321-b2ba-45eb-bea1-00978427a7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:27.721061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519d321-b2ba-45eb-bea1-00978427a7cb admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:27.722117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519d321-b2ba-45eb-bea1-00978427a7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:27.722117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519d321-b2ba-45eb-bea1-00978427a7cb admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:27.722692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519d321-b2ba-45eb-bea1-00978427a7cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:27.725477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519d321-b2ba-45eb-bea1-00978427a7cb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:27.758698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1018/2045] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:12:27 2022] POST /identity/v3/users => generated 307 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:12:27.762747 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900176a-c3ce-405c-9d9a-7c259a2cca58 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:27.777345 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900176a-c3ce-405c-9d9a-7c259a2cca58 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:27.797688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977dd9ac-8f7d-4e5a-a5ec-1f7ce335cd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:27.798375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dd9ac-8f7d-4e5a-a5ec-1f7ce335cd0b admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:27.798665 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dd9ac-8f7d-4e5a-a5ec-1f7ce335cd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:27.798941 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dd9ac-8f7d-4e5a-a5ec-1f7ce335cd0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:27.800437 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977dd9ac-8f7d-4e5a-a5ec-1f7ce335cd0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:27.802337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977dd9ac-8f7d-4e5a-a5ec-1f7ce335cd0b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:27.816543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1028/2046] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:12:27.821471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519d321-b2ba-45eb-bea1-00978427a7cb admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:27.834548 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519d321-b2ba-45eb-bea1-00978427a7cb admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:27.842371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6463d-9a0a-4152-b6b0-dc6e80e12414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:27.843407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6463d-9a0a-4152-b6b0-dc6e80e12414 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:27.843818 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6463d-9a0a-4152-b6b0-dc6e80e12414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:27.844202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6463d-9a0a-4152-b6b0-dc6e80e12414 admin admin] PATH_INFO: `/v3/projects/8561caaec5e246c59c5edc0b7826923f/users/fe5be2768bd34c76905ec39767e28367/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:27.867574 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6463d-9a0a-4152-b6b0-dc6e80e12414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8561caaec5e246c59c5edc0b7826923f, user_id=fe5be2768bd34c76905ec39767e28367, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:27.875521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6463d-9a0a-4152-b6b0-dc6e80e12414 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:27.894981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1019/2047] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:12:27 2022] PUT /identity/v3/projects/8561caaec5e246c59c5edc0b7826923f/users/fe5be2768bd34c76905ec39767e28367/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:12:27.900423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977dd9ac-8f7d-4e5a-a5ec-1f7ce335cd0b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:27.922300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977dd9ac-8f7d-4e5a-a5ec-1f7ce335cd0b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:28.055231 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cbb1a-1b08-4cf3-aeeb-e079c3533f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:28.056151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cbb1a-1b08-4cf3-aeeb-e079c3533f67 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:28.057471 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cbb1a-1b08-4cf3-aeeb-e079c3533f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:28.057887 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cbb1a-1b08-4cf3-aeeb-e079c3533f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:28.060540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cbb1a-1b08-4cf3-aeeb-e079c3533f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:28.067815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cbb1a-1b08-4cf3-aeeb-e079c3533f67 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:28.092035 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1029/2048] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:27 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:12:28.100023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6463d-9a0a-4152-b6b0-dc6e80e12414 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:28.115878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6463d-9a0a-4152-b6b0-dc6e80e12414 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:28.145303 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2438cd-f6dd-43e7-bcdb-10a57a1be783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:28.148554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2438cd-f6dd-43e7-bcdb-10a57a1be783 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:28.149057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2438cd-f6dd-43e7-bcdb-10a57a1be783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:28.149565 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2438cd-f6dd-43e7-bcdb-10a57a1be783 admin admin] PATH_INFO: `/v3/projects/8561caaec5e246c59c5edc0b7826923f/users/fe5be2768bd34c76905ec39767e28367/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:28.155352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2438cd-f6dd-43e7-bcdb-10a57a1be783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8561caaec5e246c59c5edc0b7826923f, user_id=fe5be2768bd34c76905ec39767e28367, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:28.172175 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2438cd-f6dd-43e7-bcdb-10a57a1be783 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:28.260045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1020/2049] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:12:28 2022] PUT /identity/v3/projects/8561caaec5e246c59c5edc0b7826923f/users/fe5be2768bd34c76905ec39767e28367/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:29.168104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a2dff-0727-4588-8e12-9269b40ec034 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:29.172217 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a2dff-0727-4588-8e12-9269b40ec034 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:29.174349 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a2dff-0727-4588-8e12-9269b40ec034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:29.226904 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:29.249576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4c5a2dff-0727-4588-8e12-9269b40ec034 None None] MFA Rules not processed for user `326bf42fe8c642b4898f2c8380733599`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:29.252659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4c5a2dff-0727-4588-8e12-9269b40ec034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:29.381937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1030/2050] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:29.391735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2438cd-f6dd-43e7-bcdb-10a57a1be783 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:29.394521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3d2438cd-f6dd-43e7-bcdb-10a57a1be783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:29.516891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2438cd-f6dd-43e7-bcdb-10a57a1be783 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:29.527374 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d0900-f95a-4119-9596-f35ad6adaaa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:29.527965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d0900-f95a-4119-9596-f35ad6adaaa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:29.528245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d0900-f95a-4119-9596-f35ad6adaaa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:29.528531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d0900-f95a-4119-9596-f35ad6adaaa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:29.540439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d0900-f95a-4119-9596-f35ad6adaaa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:29.541952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d0900-f95a-4119-9596-f35ad6adaaa8 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:29.558539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1021/2051] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:12:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:12:29.578773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a76dd-c1ff-4663-b576-18d3f61ba54d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:29.579048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a76dd-c1ff-4663-b576-18d3f61ba54d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:29.579334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a76dd-c1ff-4663-b576-18d3f61ba54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:29.624953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-9a2a76dd-c1ff-4663-b576-18d3f61ba54d None None] MFA Rules not processed for user `fe5be2768bd34c76905ec39767e28367`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:29.626978 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9a2a76dd-c1ff-4663-b576-18d3f61ba54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:29.827219 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1031/2052] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:29 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:32.162652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d0900-f95a-4119-9596-f35ad6adaaa8 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:32.263720 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d0900-f95a-4119-9596-f35ad6adaaa8 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:32.270756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e5c68-0e14-4032-a08d-eff75dc71f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:32.271954 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e5c68-0e14-4032-a08d-eff75dc71f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:32.272248 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e5c68-0e14-4032-a08d-eff75dc71f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:32.272728 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e5c68-0e14-4032-a08d-eff75dc71f98 admin admin] PATH_INFO: `/v3/users/13439b8b7a5b4a3f9038fcecadc2d03f` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:32.274483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e5c68-0e14-4032-a08d-eff75dc71f98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13439b8b7a5b4a3f9038fcecadc2d03f)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:32.276923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e5c68-0e14-4032-a08d-eff75dc71f98 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:32.347439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:32.391261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c85e5c68-0e14-4032-a08d-eff75dc71f98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13439b8b7a5b4a3f9038fcecadc2d03f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:32.412177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c85e5c68-0e14-4032-a08d-eff75dc71f98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13439b8b7a5b4a3f9038fcecadc2d03f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:32.431824 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c85e5c68-0e14-4032-a08d-eff75dc71f98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13439b8b7a5b4a3f9038fcecadc2d03f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:32.470998 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c85e5c68-0e14-4032-a08d-eff75dc71f98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13439b8b7a5b4a3f9038fcecadc2d03f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:32.474880 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c85e5c68-0e14-4032-a08d-eff75dc71f98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13439b8b7a5b4a3f9038fcecadc2d03f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:32.504253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1022/2053] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:12:32 2022] DELETE /identity/v3/users/13439b8b7a5b4a3f9038fcecadc2d03f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:32.528146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5b37c-166d-49f4-b6e7-c1fc330a11d7 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:32.528924 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5b37c-166d-49f4-b6e7-c1fc330a11d7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:32.528924 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5b37c-166d-49f4-b6e7-c1fc330a11d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:32.559377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c7c5b37c-166d-49f4-b6e7-c1fc330a11d7 None None] MFA Rules not processed for user `326bf42fe8c642b4898f2c8380733599`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:32.560987 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c7c5b37c-166d-49f4-b6e7-c1fc330a11d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:32.725374 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1032/2054] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:12:32.938432 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e5c68-0e14-4032-a08d-eff75dc71f98 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:32.940062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c85e5c68-0e14-4032-a08d-eff75dc71f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:33.030932 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e5c68-0e14-4032-a08d-eff75dc71f98 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:33.037576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349570a-764c-4f30-81af-b3dccc042f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:33.038861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349570a-764c-4f30-81af-b3dccc042f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:33.038861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349570a-764c-4f30-81af-b3dccc042f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:33.039156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349570a-764c-4f30-81af-b3dccc042f60 admin admin] PATH_INFO: `/v3/projects/2cbd7fa520874db497a00b3ab41acee6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:33.040438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349570a-764c-4f30-81af-b3dccc042f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cbd7fa520874db497a00b3ab41acee6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:33.042407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349570a-764c-4f30-81af-b3dccc042f60 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:33.106827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e349570a-764c-4f30-81af-b3dccc042f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cbd7fa520874db497a00b3ab41acee6', 'request_id': 'req-e349570a-764c-4f30-81af-b3dccc042f60'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:33.117073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e349570a-764c-4f30-81af-b3dccc042f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cbd7fa520874db497a00b3ab41acee6', 'request_id': 'req-e349570a-764c-4f30-81af-b3dccc042f60'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:33.129139 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e349570a-764c-4f30-81af-b3dccc042f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cbd7fa520874db497a00b3ab41acee6', 'request_id': 'req-e349570a-764c-4f30-81af-b3dccc042f60'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:33.147690 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e349570a-764c-4f30-81af-b3dccc042f60 admin admin] The token cache is being invalidate because project 2cbd7fa520874db497a00b3ab41acee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:12:33.148641 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e349570a-764c-4f30-81af-b3dccc042f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:33.149576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e349570a-764c-4f30-81af-b3dccc042f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:33.155471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1023/2055] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:12:32 2022] DELETE /identity/v3/projects/2cbd7fa520874db497a00b3ab41acee6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:33.159343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5b37c-166d-49f4-b6e7-c1fc330a11d7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:33.182275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5b37c-166d-49f4-b6e7-c1fc330a11d7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:33.223379 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:33.273274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f82ec4-aee6-4d27-9870-206be22d8263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:33.274594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f82ec4-aee6-4d27-9870-206be22d8263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:33.275232 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f82ec4-aee6-4d27-9870-206be22d8263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:33.275709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f82ec4-aee6-4d27-9870-206be22d8263 admin admin] PATH_INFO: `/v3/users/fe5be2768bd34c76905ec39767e28367` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:33.277682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f82ec4-aee6-4d27-9870-206be22d8263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe5be2768bd34c76905ec39767e28367)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:33.280150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f82ec4-aee6-4d27-9870-206be22d8263 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:33.362701 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b9f82ec4-aee6-4d27-9870-206be22d8263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe5be2768bd34c76905ec39767e28367'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:33.363609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b9f82ec4-aee6-4d27-9870-206be22d8263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe5be2768bd34c76905ec39767e28367'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:33.378509 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b9f82ec4-aee6-4d27-9870-206be22d8263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe5be2768bd34c76905ec39767e28367'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:33.408096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b9f82ec4-aee6-4d27-9870-206be22d8263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe5be2768bd34c76905ec39767e28367'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:33.422233 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b9f82ec4-aee6-4d27-9870-206be22d8263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe5be2768bd34c76905ec39767e28367'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:33.436729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1033/2056] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:12:33 2022] DELETE /identity/v3/users/fe5be2768bd34c76905ec39767e28367 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:33.738250 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349570a-764c-4f30-81af-b3dccc042f60 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:33.749295 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e349570a-764c-4f30-81af-b3dccc042f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:33.804343 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349570a-764c-4f30-81af-b3dccc042f60 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:33.902823 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:33.947816 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfac789-d401-425e-b76a-ced8a60272bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:33.948848 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfac789-d401-425e-b76a-ced8a60272bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:33.949212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfac789-d401-425e-b76a-ced8a60272bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:33.949570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfac789-d401-425e-b76a-ced8a60272bc admin admin] PATH_INFO: `/v3/projects/8561caaec5e246c59c5edc0b7826923f` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:33.951424 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfac789-d401-425e-b76a-ced8a60272bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8561caaec5e246c59c5edc0b7826923f)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:33.953549 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfac789-d401-425e-b76a-ced8a60272bc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:34.025375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2cfac789-d401-425e-b76a-ced8a60272bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8561caaec5e246c59c5edc0b7826923f', 'request_id': 'req-2cfac789-d401-425e-b76a-ced8a60272bc'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:34.032782 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2cfac789-d401-425e-b76a-ced8a60272bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8561caaec5e246c59c5edc0b7826923f', 'request_id': 'req-2cfac789-d401-425e-b76a-ced8a60272bc'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:34.053005 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2cfac789-d401-425e-b76a-ced8a60272bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8561caaec5e246c59c5edc0b7826923f', 'request_id': 'req-2cfac789-d401-425e-b76a-ced8a60272bc'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:34.070456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2cfac789-d401-425e-b76a-ced8a60272bc admin admin] The token cache is being invalidate because project 8561caaec5e246c59c5edc0b7826923f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:12:34.070897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2cfac789-d401-425e-b76a-ced8a60272bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:34.072493 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2cfac789-d401-425e-b76a-ced8a60272bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:34.075685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1024/2057] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:12:33 2022] DELETE /identity/v3/projects/8561caaec5e246c59c5edc0b7826923f => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:34.081830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc4e81-5faf-484b-a699-5ee185f30879 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:34.082698 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc4e81-5faf-484b-a699-5ee185f30879 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:34.082989 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc4e81-5faf-484b-a699-5ee185f30879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:34.122386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-66fc4e81-5faf-484b-a699-5ee185f30879 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:34.132762 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-66fc4e81-5faf-484b-a699-5ee185f30879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:34.283401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1034/2058] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:34.292783 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ef8d0-1cf4-4156-8a42-b8ca19ceaf9e None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:34.298761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ef8d0-1cf4-4156-8a42-b8ca19ceaf9e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:34.299477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ef8d0-1cf4-4156-8a42-b8ca19ceaf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:34.344867 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-cc1ef8d0-1cf4-4156-8a42-b8ca19ceaf9e None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:34.348229 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-cc1ef8d0-1cf4-4156-8a42-b8ca19ceaf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:34.367479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1025/2059] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:34.372138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc4e81-5faf-484b-a699-5ee185f30879 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:34.389915 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc4e81-5faf-484b-a699-5ee185f30879 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:34.405335 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db888812-004c-4232-b900-98b7050c7823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:34.406125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db888812-004c-4232-b900-98b7050c7823 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:34.406555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db888812-004c-4232-b900-98b7050c7823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:34.407043 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db888812-004c-4232-b900-98b7050c7823 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:34.407943 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db888812-004c-4232-b900-98b7050c7823 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:12:34.408380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db888812-004c-4232-b900-98b7050c7823 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:34.410722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db888812-004c-4232-b900-98b7050c7823 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:34.433714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1035/2060] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 09:12:34.435167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ef8d0-1cf4-4156-8a42-b8ca19ceaf9e None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:34.471720 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ef8d0-1cf4-4156-8a42-b8ca19ceaf9e None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:34.492047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4312a3-ec7d-4e27-8fcf-8d1acd3978c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:34.492755 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4312a3-ec7d-4e27-8fcf-8d1acd3978c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:34.493039 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4312a3-ec7d-4e27-8fcf-8d1acd3978c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:34.493322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4312a3-ec7d-4e27-8fcf-8d1acd3978c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:34.494054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4312a3-ec7d-4e27-8fcf-8d1acd3978c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:34.498164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4312a3-ec7d-4e27-8fcf-8d1acd3978c8 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:34.558405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db888812-004c-4232-b900-98b7050c7823 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:34.559532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1026/2061] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:12:34 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:12:34.592868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db888812-004c-4232-b900-98b7050c7823 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:34.715687 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:34.800832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f423fe7-17bc-4236-8ef9-f0be7a4988f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:34.802845 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f423fe7-17bc-4236-8ef9-f0be7a4988f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:34.803373 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f423fe7-17bc-4236-8ef9-f0be7a4988f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:34.804027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f423fe7-17bc-4236-8ef9-f0be7a4988f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:34.806786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f423fe7-17bc-4236-8ef9-f0be7a4988f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:34.811548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f423fe7-17bc-4236-8ef9-f0be7a4988f5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:34.871847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1036/2062] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:12:34 2022] POST /identity/v3/users => generated 287 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:12:34.875790 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4312a3-ec7d-4e27-8fcf-8d1acd3978c8 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:34.905720 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4312a3-ec7d-4e27-8fcf-8d1acd3978c8 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:34.919232 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cacafc-d42b-4fa8-ad19-4e673a009531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:34.920268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cacafc-d42b-4fa8-ad19-4e673a009531 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:34.920556 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cacafc-d42b-4fa8-ad19-4e673a009531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:34.920836 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cacafc-d42b-4fa8-ad19-4e673a009531 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:34.921601 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cacafc-d42b-4fa8-ad19-4e673a009531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:34.923442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cacafc-d42b-4fa8-ad19-4e673a009531 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:34.940398 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1027/2063] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:12:34.942078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f423fe7-17bc-4236-8ef9-f0be7a4988f5 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:34.960202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f423fe7-17bc-4236-8ef9-f0be7a4988f5 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:34.975240 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbf923-ccce-475e-a311-73b27d3f2de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:34.976620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbf923-ccce-475e-a311-73b27d3f2de4 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:34.977837 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbf923-ccce-475e-a311-73b27d3f2de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:34.978375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbf923-ccce-475e-a311-73b27d3f2de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:34.979527 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efbf923-ccce-475e-a311-73b27d3f2de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:34.982193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efbf923-ccce-475e-a311-73b27d3f2de4 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:35.008291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1037/2064] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:12:35.013165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cacafc-d42b-4fa8-ad19-4e673a009531 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:35.029987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cacafc-d42b-4fa8-ad19-4e673a009531 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:35.045935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a7352-dec8-47a0-ac69-41910e85318f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:35.046861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a7352-dec8-47a0-ac69-41910e85318f admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:35.047252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a7352-dec8-47a0-ac69-41910e85318f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:35.047406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a7352-dec8-47a0-ac69-41910e85318f admin admin] PATH_INFO: `/v3/projects/3956d6c47b0e464fbbc4b30bd78326ef/users/6279105a28b343bf9eb89c3ef5c74361/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:35.085837 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a7352-dec8-47a0-ac69-41910e85318f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3956d6c47b0e464fbbc4b30bd78326ef, user_id=6279105a28b343bf9eb89c3ef5c74361, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:35.088983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a7352-dec8-47a0-ac69-41910e85318f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:35.158200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1028/2065] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:12:35 2022] PUT /identity/v3/projects/3956d6c47b0e464fbbc4b30bd78326ef/users/6279105a28b343bf9eb89c3ef5c74361/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:12:35.170310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77038e03-160e-4297-8a89-c8883a6205ad None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:35.171355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77038e03-160e-4297-8a89-c8883a6205ad None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:35.171964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77038e03-160e-4297-8a89-c8883a6205ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:35.215140 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-77038e03-160e-4297-8a89-c8883a6205ad None None] MFA Rules not processed for user `6279105a28b343bf9eb89c3ef5c74361`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:35.217338 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-77038e03-160e-4297-8a89-c8883a6205ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:35.347238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd0902-633a-4fd6-82e7-3d8960bddbb0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:35.347744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd0902-633a-4fd6-82e7-3d8960bddbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:35.348249 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd0902-633a-4fd6-82e7-3d8960bddbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:35.384459 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1038/2066] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:35.391156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77038e03-160e-4297-8a89-c8883a6205ad None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:35.413244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ebcd0902-633a-4fd6-82e7-3d8960bddbb0 None None] MFA Rules not processed for user `83a97e1d803043acadb43266a0bc0cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:35.415634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ebcd0902-633a-4fd6-82e7-3d8960bddbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:35.500154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77038e03-160e-4297-8a89-c8883a6205ad None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:35.509368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05137a6-1153-4e20-a0e2-df8025342ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:35.510465 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05137a6-1153-4e20-a0e2-df8025342ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:35.511051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05137a6-1153-4e20-a0e2-df8025342ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:35.511722 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05137a6-1153-4e20-a0e2-df8025342ba5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:35.513498 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05137a6-1153-4e20-a0e2-df8025342ba5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:35.517350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05137a6-1153-4e20-a0e2-df8025342ba5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:35.541887 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:35.591132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1039/2067] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:12:35 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:12:35.597936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05137a6-1153-4e20-a0e2-df8025342ba5 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:35.615970 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1029/2068] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:35 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:12:35.642389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd0902-633a-4fd6-82e7-3d8960bddbb0 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:35.644509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ebcd0902-633a-4fd6-82e7-3d8960bddbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:35.686012 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:35.782793 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05137a6-1153-4e20-a0e2-df8025342ba5 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:35.792864 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a718e-534d-4157-bb8b-acf3a2f44245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:35.794224 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a718e-534d-4157-bb8b-acf3a2f44245 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:35.794783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a718e-534d-4157-bb8b-acf3a2f44245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:35.795261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a718e-534d-4157-bb8b-acf3a2f44245 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:35.797346 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a718e-534d-4157-bb8b-acf3a2f44245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:35.803586 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a718e-534d-4157-bb8b-acf3a2f44245 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:35.845921 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd0902-633a-4fd6-82e7-3d8960bddbb0 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:35.861178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1040/2069] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:12:35 2022] POST /identity/v3/users => generated 293 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:12:35.868730 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a718e-534d-4157-bb8b-acf3a2f44245 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:35.895267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a718e-534d-4157-bb8b-acf3a2f44245 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:35.906741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68822e0d-96e3-4e11-8682-193b12a70477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:35.908073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68822e0d-96e3-4e11-8682-193b12a70477 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:35.909645 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68822e0d-96e3-4e11-8682-193b12a70477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:35.911008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68822e0d-96e3-4e11-8682-193b12a70477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:35.913595 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68822e0d-96e3-4e11-8682-193b12a70477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:35.917765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68822e0d-96e3-4e11-8682-193b12a70477 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:35.952620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1041/2070] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:12:35.956850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68822e0d-96e3-4e11-8682-193b12a70477 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:35.985556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68822e0d-96e3-4e11-8682-193b12a70477 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:35.996026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cc0be-d64d-4f38-aa29-03d85a24aab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:35.997348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cc0be-d64d-4f38-aa29-03d85a24aab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:35.998262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cc0be-d64d-4f38-aa29-03d85a24aab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:35.998655 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cc0be-d64d-4f38-aa29-03d85a24aab2 admin admin] PATH_INFO: `/v3/projects/16ae59b0600745c3838472e96815df05/users/924ff3e52ea0423eaa1f036750ae777e/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:36.017547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b5ccd-2420-4264-aa4a-084e5cbf92bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:36.018223 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b5ccd-2420-4264-aa4a-084e5cbf92bc service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:36.018509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b5ccd-2420-4264-aa4a-084e5cbf92bc service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:36.018970 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b5ccd-2420-4264-aa4a-084e5cbf92bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:36.021684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-953b5ccd-2420-4264-aa4a-084e5cbf92bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:36.056417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cc0be-d64d-4f38-aa29-03d85a24aab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16ae59b0600745c3838472e96815df05, user_id=924ff3e52ea0423eaa1f036750ae777e, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:36.060512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cc0be-d64d-4f38-aa29-03d85a24aab2 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:36.087960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1042/2071] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:12:35 2022] PUT /identity/v3/projects/16ae59b0600745c3838472e96815df05/users/924ff3e52ea0423eaa1f036750ae777e/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:36.090650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cc0be-d64d-4f38-aa29-03d85a24aab2 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:36.155024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b5ccd-2420-4264-aa4a-084e5cbf92bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:36.156734 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b5ccd-2420-4264-aa4a-084e5cbf92bc service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:36.175371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1030/2072] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:12:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:12:36.222236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cc0be-d64d-4f38-aa29-03d85a24aab2 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:36.235504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead27ca8-3b48-41c2-9b79-5cfebcc6e8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:36.236815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead27ca8-3b48-41c2-9b79-5cfebcc6e8cb admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:36.237920 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead27ca8-3b48-41c2-9b79-5cfebcc6e8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:36.238394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead27ca8-3b48-41c2-9b79-5cfebcc6e8cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:36.242560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead27ca8-3b48-41c2-9b79-5cfebcc6e8cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:36.248395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead27ca8-3b48-41c2-9b79-5cfebcc6e8cb admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:36.272543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1043/2073] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:12:36.278862 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b5ccd-2420-4264-aa4a-084e5cbf92bc service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:36.303561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b5ccd-2420-4264-aa4a-084e5cbf92bc service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:36.313184 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918423a4-303a-4937-9831-7c27d1bc5fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:36.314768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918423a4-303a-4937-9831-7c27d1bc5fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:36.315288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918423a4-303a-4937-9831-7c27d1bc5fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:36.315810 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918423a4-303a-4937-9831-7c27d1bc5fe5 admin admin] PATH_INFO: `/v3/projects/16ae59b0600745c3838472e96815df05/users/924ff3e52ea0423eaa1f036750ae777e/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:36.320264 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918423a4-303a-4937-9831-7c27d1bc5fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16ae59b0600745c3838472e96815df05, user_id=924ff3e52ea0423eaa1f036750ae777e, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:36.340544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918423a4-303a-4937-9831-7c27d1bc5fe5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:36.396495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1031/2074] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:12:36 2022] PUT /identity/v3/projects/16ae59b0600745c3838472e96815df05/users/924ff3e52ea0423eaa1f036750ae777e/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:36.422898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fdcd2-d8b7-459a-b273-bb0c954f85e8 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:36.423240 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fdcd2-d8b7-459a-b273-bb0c954f85e8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:36.423661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fdcd2-d8b7-459a-b273-bb0c954f85e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:36.483637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1e7fdcd2-d8b7-459a-b273-bb0c954f85e8 None None] MFA Rules not processed for user `924ff3e52ea0423eaa1f036750ae777e`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:36.485023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1e7fdcd2-d8b7-459a-b273-bb0c954f85e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:36.634378 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1044/2075] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:12:36.652846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918423a4-303a-4937-9831-7c27d1bc5fe5 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:36.655598 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-918423a4-303a-4937-9831-7c27d1bc5fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:36.680335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:36.902312 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918423a4-303a-4937-9831-7c27d1bc5fe5 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:36.907825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ecf16-8ad3-44ef-8a79-e6fdc1ae8645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:36.908816 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ecf16-8ad3-44ef-8a79-e6fdc1ae8645 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:36.909987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ecf16-8ad3-44ef-8a79-e6fdc1ae8645 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:36.910376 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ecf16-8ad3-44ef-8a79-e6fdc1ae8645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:36.928129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ecf16-8ad3-44ef-8a79-e6fdc1ae8645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:36.929887 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ecf16-8ad3-44ef-8a79-e6fdc1ae8645 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:36.949917 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1032/2076] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:12:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:12:37.091539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fdcd2-d8b7-459a-b273-bb0c954f85e8 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:37.124132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fdcd2-d8b7-459a-b273-bb0c954f85e8 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:37.137004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720f9a9-adef-4c73-b9c0-dafce97e42a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:37.137691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720f9a9-adef-4c73-b9c0-dafce97e42a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:37.138072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720f9a9-adef-4c73-b9c0-dafce97e42a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:37.138226 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720f9a9-adef-4c73-b9c0-dafce97e42a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:37.177977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:37.394018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720f9a9-adef-4c73-b9c0-dafce97e42a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:37.395525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720f9a9-adef-4c73-b9c0-dafce97e42a3 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:37.411317 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1045/2077] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:12:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:12:46.937311 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=22 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:47.406240 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=25 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:51.418743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ecf16-8ad3-44ef-8a79-e6fdc1ae8645 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:51.420369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bd2ecf16-8ad3-44ef-8a79-e6fdc1ae8645 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:51.494736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ecf16-8ad3-44ef-8a79-e6fdc1ae8645 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:51.499763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7326e7-2b34-4384-9403-65385c6143d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:51.500702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7326e7-2b34-4384-9403-65385c6143d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:51.501100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7326e7-2b34-4384-9403-65385c6143d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:51.501381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7326e7-2b34-4384-9403-65385c6143d8 admin admin] PATH_INFO: `/v3/users/5408e0c011514976a6d7b51823deb563` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:51.502738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7326e7-2b34-4384-9403-65385c6143d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5408e0c011514976a6d7b51823deb563)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:51.506704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7326e7-2b34-4384-9403-65385c6143d8 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:51.565576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8e7326e7-2b34-4384-9403-65385c6143d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5408e0c011514976a6d7b51823deb563'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:51.576823 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8e7326e7-2b34-4384-9403-65385c6143d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5408e0c011514976a6d7b51823deb563'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:51.594117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8e7326e7-2b34-4384-9403-65385c6143d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5408e0c011514976a6d7b51823deb563'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:51.613124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8e7326e7-2b34-4384-9403-65385c6143d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5408e0c011514976a6d7b51823deb563'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:51.613911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8e7326e7-2b34-4384-9403-65385c6143d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5408e0c011514976a6d7b51823deb563'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:51.622515 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1033/2078] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:12:51 2022] DELETE /identity/v3/users/5408e0c011514976a6d7b51823deb563 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:51.628109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c24f80-a67e-4b7d-a508-f2b3d99f025a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:51.628840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c24f80-a67e-4b7d-a508-f2b3d99f025a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:51.629146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c24f80-a67e-4b7d-a508-f2b3d99f025a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:51.653638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-33c24f80-a67e-4b7d-a508-f2b3d99f025a None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:51.655085 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-33c24f80-a67e-4b7d-a508-f2b3d99f025a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:51.757630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1046/2079] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:51.769446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7326e7-2b34-4384-9403-65385c6143d8 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:51.772404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8e7326e7-2b34-4384-9403-65385c6143d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:51.900637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7326e7-2b34-4384-9403-65385c6143d8 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:51.904523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7eb93-a2c6-4d5f-8261-1d397ea595da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:51.905235 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7eb93-a2c6-4d5f-8261-1d397ea595da service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:51.905560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7eb93-a2c6-4d5f-8261-1d397ea595da service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:51.905878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7eb93-a2c6-4d5f-8261-1d397ea595da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:51.918024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7eb93-a2c6-4d5f-8261-1d397ea595da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:51.919532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7eb93-a2c6-4d5f-8261-1d397ea595da service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:51.932112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1034/2080] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:12:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:12:52.569440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c24f80-a67e-4b7d-a508-f2b3d99f025a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:52.571927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-33c24f80-a67e-4b7d-a508-f2b3d99f025a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:52.587046 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c24f80-a67e-4b7d-a508-f2b3d99f025a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:52.597718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d4193-69aa-49c6-ac37-331199e6846a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:52.599482 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d4193-69aa-49c6-ac37-331199e6846a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:52.600359 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d4193-69aa-49c6-ac37-331199e6846a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:52.600946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d4193-69aa-49c6-ac37-331199e6846a admin admin] PATH_INFO: `/v3/projects/7237d9a716224e1aac9f1277211fc998` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:52.605201 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d4193-69aa-49c6-ac37-331199e6846a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7237d9a716224e1aac9f1277211fc998)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:52.609247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d4193-69aa-49c6-ac37-331199e6846a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:52.690883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e05d4193-69aa-49c6-ac37-331199e6846a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7237d9a716224e1aac9f1277211fc998', 'request_id': 'req-e05d4193-69aa-49c6-ac37-331199e6846a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:52.698397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e05d4193-69aa-49c6-ac37-331199e6846a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7237d9a716224e1aac9f1277211fc998', 'request_id': 'req-e05d4193-69aa-49c6-ac37-331199e6846a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:52.717609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e05d4193-69aa-49c6-ac37-331199e6846a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7237d9a716224e1aac9f1277211fc998', 'request_id': 'req-e05d4193-69aa-49c6-ac37-331199e6846a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:52.736894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e05d4193-69aa-49c6-ac37-331199e6846a admin admin] The token cache is being invalidate because project 7237d9a716224e1aac9f1277211fc998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:12:52.737360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e05d4193-69aa-49c6-ac37-331199e6846a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:52.738442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e05d4193-69aa-49c6-ac37-331199e6846a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:12:52.741852 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1047/2081] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:12:52 2022] DELETE /identity/v3/projects/7237d9a716224e1aac9f1277211fc998 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:52.750372 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be7d68-6856-476a-beef-ec737425d19b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:52.751767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be7d68-6856-476a-beef-ec737425d19b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:52.752330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be7d68-6856-476a-beef-ec737425d19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:52.788937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c5be7d68-6856-476a-beef-ec737425d19b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:52.791075 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c5be7d68-6856-476a-beef-ec737425d19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:52.825131 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:52.895201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1035/2082] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:52.904130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c4670-69d1-4ca8-9715-90126fa6a460 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:52.904449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c4670-69d1-4ca8-9715-90126fa6a460 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:52.904738 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c4670-69d1-4ca8-9715-90126fa6a460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:52.925473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ed5c4670-69d1-4ca8-9715-90126fa6a460 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:52.927476 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ed5c4670-69d1-4ca8-9715-90126fa6a460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:52.936621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1048/2083] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:52.941103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be7d68-6856-476a-beef-ec737425d19b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:52.959357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be7d68-6856-476a-beef-ec737425d19b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:52.964849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccecfdc-bda5-4c77-bd98-6af99a6aa8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:52.965568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccecfdc-bda5-4c77-bd98-6af99a6aa8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:52.965889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccecfdc-bda5-4c77-bd98-6af99a6aa8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:52.966199 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccecfdc-bda5-4c77-bd98-6af99a6aa8a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:52.966990 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccecfdc-bda5-4c77-bd98-6af99a6aa8a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:12:52.967338 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccecfdc-bda5-4c77-bd98-6af99a6aa8a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:52.969162 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccecfdc-bda5-4c77-bd98-6af99a6aa8a3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:52.987972 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1036/2084] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 09:12:52.993683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c4670-69d1-4ca8-9715-90126fa6a460 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:53.011765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c4670-69d1-4ca8-9715-90126fa6a460 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:53.019879 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65561607-1629-4b4b-a14b-37adfba730e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:53.021197 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65561607-1629-4b4b-a14b-37adfba730e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:53.021686 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65561607-1629-4b4b-a14b-37adfba730e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:53.022172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65561607-1629-4b4b-a14b-37adfba730e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:53.023493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65561607-1629-4b4b-a14b-37adfba730e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:53.027490 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65561607-1629-4b4b-a14b-37adfba730e6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:53.076900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1049/2085] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:12:52 2022] POST /identity/v3/projects => generated 397 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:12:53.082720 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccecfdc-bda5-4c77-bd98-6af99a6aa8a3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:53.096202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccecfdc-bda5-4c77-bd98-6af99a6aa8a3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:53.165887 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97b96e-3c09-4961-a92c-3c0ee4309769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:53.166566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97b96e-3c09-4961-a92c-3c0ee4309769 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:53.166852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97b96e-3c09-4961-a92c-3c0ee4309769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:53.167136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97b96e-3c09-4961-a92c-3c0ee4309769 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:53.167830 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97b96e-3c09-4961-a92c-3c0ee4309769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:53.169839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97b96e-3c09-4961-a92c-3c0ee4309769 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:53.199235 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1037/2086] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:12:53 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:12:53.203554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65561607-1629-4b4b-a14b-37adfba730e6 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:53.216580 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65561607-1629-4b4b-a14b-37adfba730e6 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:53.222170 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643e702-6294-4d14-8570-788de46e4653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:53.222986 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643e702-6294-4d14-8570-788de46e4653 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:53.224287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643e702-6294-4d14-8570-788de46e4653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:53.225108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643e702-6294-4d14-8570-788de46e4653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:53.226012 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643e702-6294-4d14-8570-788de46e4653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:53.227858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643e702-6294-4d14-8570-788de46e4653 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:53.240287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1050/2087] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:12:53.245266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97b96e-3c09-4961-a92c-3c0ee4309769 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:53.261729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97b96e-3c09-4961-a92c-3c0ee4309769 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:53.267412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463136e0-607d-44c2-a692-24777a007b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:53.268327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463136e0-607d-44c2-a692-24777a007b33 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:53.268764 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463136e0-607d-44c2-a692-24777a007b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:53.269389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463136e0-607d-44c2-a692-24777a007b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:53.271664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463136e0-607d-44c2-a692-24777a007b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:53.274497 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463136e0-607d-44c2-a692-24777a007b33 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:53.292768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1038/2088] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:12:53.297662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643e702-6294-4d14-8570-788de46e4653 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:53.313843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643e702-6294-4d14-8570-788de46e4653 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:53.323909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c65360-a877-405a-bf55-ba6fe28f6549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:53.325470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c65360-a877-405a-bf55-ba6fe28f6549 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:53.325746 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c65360-a877-405a-bf55-ba6fe28f6549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:53.326022 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c65360-a877-405a-bf55-ba6fe28f6549 admin admin] PATH_INFO: `/v3/projects/0afe9b59952a4b83aa1e8c5c6b3bedad/users/c33e5ba88fc94738a81d7452f1bdf94d/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:53.349913 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c65360-a877-405a-bf55-ba6fe28f6549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0afe9b59952a4b83aa1e8c5c6b3bedad, user_id=c33e5ba88fc94738a81d7452f1bdf94d, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:53.352929 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c65360-a877-405a-bf55-ba6fe28f6549 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:53.371973 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1051/2089] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:12:53 2022] PUT /identity/v3/projects/0afe9b59952a4b83aa1e8c5c6b3bedad/users/c33e5ba88fc94738a81d7452f1bdf94d/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:12:53.382723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c8dec-f846-4534-9503-698d0d7841e4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:53.383026 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c8dec-f846-4534-9503-698d0d7841e4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:53.383286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c8dec-f846-4534-9503-698d0d7841e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:53.417253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c65c8dec-f846-4534-9503-698d0d7841e4 None None] MFA Rules not processed for user `c33e5ba88fc94738a81d7452f1bdf94d`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:12:53.427273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c65c8dec-f846-4534-9503-698d0d7841e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:53.519115 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1039/2090] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:12:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:12:53.556940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c65360-a877-405a-bf55-ba6fe28f6549 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:53.559961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-25c65360-a877-405a-bf55-ba6fe28f6549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:53.609089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:53.660761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c65360-a877-405a-bf55-ba6fe28f6549 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:53.663441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30ef94-2d13-4506-8d93-a2a771cf899e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:53.664175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30ef94-2d13-4506-8d93-a2a771cf899e service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:53.664463 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30ef94-2d13-4506-8d93-a2a771cf899e service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:53.664744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30ef94-2d13-4506-8d93-a2a771cf899e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:53.679503 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe30ef94-2d13-4506-8d93-a2a771cf899e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:53.681964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe30ef94-2d13-4506-8d93-a2a771cf899e service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:53.699200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1052/2091] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:12:53 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:12:55.017455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c8dec-f846-4534-9503-698d0d7841e4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:12:55.019890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c65c8dec-f846-4534-9503-698d0d7841e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:55.046013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:12:55.117834 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c8dec-f846-4534-9503-698d0d7841e4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:12:55.128414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d561e-3ddd-4e96-a28a-2d5ae6abaeae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:12:55.129119 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d561e-3ddd-4e96-a28a-2d5ae6abaeae service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:12:55.129512 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d561e-3ddd-4e96-a28a-2d5ae6abaeae service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:12:55.130721 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d561e-3ddd-4e96-a28a-2d5ae6abaeae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:12:55.132763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-616d561e-3ddd-4e96-a28a-2d5ae6abaeae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:12:55.145876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d561e-3ddd-4e96-a28a-2d5ae6abaeae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:12:55.147590 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d561e-3ddd-4e96-a28a-2d5ae6abaeae service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:12:55.168160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1040/2092] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:12:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:13:03.689016 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=17 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:04.515714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30ef94-2d13-4506-8d93-a2a771cf899e service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:04.592152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30ef94-2d13-4506-8d93-a2a771cf899e service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:04.596968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e0921-847a-4352-b967-b98140773c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:04.597850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e0921-847a-4352-b967-b98140773c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:04.598660 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e0921-847a-4352-b967-b98140773c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:04.599068 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e0921-847a-4352-b967-b98140773c16 admin admin] PATH_INFO: `/v3/users/c33e5ba88fc94738a81d7452f1bdf94d` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:04.600663 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e0921-847a-4352-b967-b98140773c16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c33e5ba88fc94738a81d7452f1bdf94d)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:04.602637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e0921-847a-4352-b967-b98140773c16 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:04.659352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ed3e0921-847a-4352-b967-b98140773c16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c33e5ba88fc94738a81d7452f1bdf94d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:04.660051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ed3e0921-847a-4352-b967-b98140773c16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c33e5ba88fc94738a81d7452f1bdf94d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:04.669144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ed3e0921-847a-4352-b967-b98140773c16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c33e5ba88fc94738a81d7452f1bdf94d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:04.695059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ed3e0921-847a-4352-b967-b98140773c16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c33e5ba88fc94738a81d7452f1bdf94d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:04.705524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ed3e0921-847a-4352-b967-b98140773c16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c33e5ba88fc94738a81d7452f1bdf94d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:04.713519 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1053/2093] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:13:04 2022] DELETE /identity/v3/users/c33e5ba88fc94738a81d7452f1bdf94d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:13:04.718340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f22469-2277-4f56-aefd-e5b3185bec57 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:04.719160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f22469-2277-4f56-aefd-e5b3185bec57 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:04.719320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f22469-2277-4f56-aefd-e5b3185bec57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:04.740297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-64f22469-2277-4f56-aefd-e5b3185bec57 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:13:04.741565 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-64f22469-2277-4f56-aefd-e5b3185bec57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:04.821093 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1041/2094] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:13:04.833226 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e0921-847a-4352-b967-b98140773c16 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:04.834876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ed3e0921-847a-4352-b967-b98140773c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:04.906025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e0921-847a-4352-b967-b98140773c16 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:04.912069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5619c2e4-5b80-4305-8c92-7c5af3c3a3b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:04.913109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619c2e4-5b80-4305-8c92-7c5af3c3a3b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:04.913590 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619c2e4-5b80-4305-8c92-7c5af3c3a3b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:04.914056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619c2e4-5b80-4305-8c92-7c5af3c3a3b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:04.929153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619c2e4-5b80-4305-8c92-7c5af3c3a3b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:04.930805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5619c2e4-5b80-4305-8c92-7c5af3c3a3b7 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:04.942927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1054/2095] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:13:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:13:05.343602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f22469-2277-4f56-aefd-e5b3185bec57 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:05.346145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-64f22469-2277-4f56-aefd-e5b3185bec57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:05.386759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f22469-2277-4f56-aefd-e5b3185bec57 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:05.393284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a728ab9-cceb-4d99-8422-2ee4ab97de26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:05.394681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a728ab9-cceb-4d99-8422-2ee4ab97de26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:05.395716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a728ab9-cceb-4d99-8422-2ee4ab97de26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:05.396243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a728ab9-cceb-4d99-8422-2ee4ab97de26 admin admin] PATH_INFO: `/v3/projects/0afe9b59952a4b83aa1e8c5c6b3bedad` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:05.398385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a728ab9-cceb-4d99-8422-2ee4ab97de26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0afe9b59952a4b83aa1e8c5c6b3bedad)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:05.401161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a728ab9-cceb-4d99-8422-2ee4ab97de26 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:05.485484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0a728ab9-cceb-4d99-8422-2ee4ab97de26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0afe9b59952a4b83aa1e8c5c6b3bedad', 'request_id': 'req-0a728ab9-cceb-4d99-8422-2ee4ab97de26'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:05.496838 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0a728ab9-cceb-4d99-8422-2ee4ab97de26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0afe9b59952a4b83aa1e8c5c6b3bedad', 'request_id': 'req-0a728ab9-cceb-4d99-8422-2ee4ab97de26'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:05.512681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0a728ab9-cceb-4d99-8422-2ee4ab97de26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0afe9b59952a4b83aa1e8c5c6b3bedad', 'request_id': 'req-0a728ab9-cceb-4d99-8422-2ee4ab97de26'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:05.531285 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0a728ab9-cceb-4d99-8422-2ee4ab97de26 admin admin] The token cache is being invalidate because project 0afe9b59952a4b83aa1e8c5c6b3bedad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:13:05.531748 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0a728ab9-cceb-4d99-8422-2ee4ab97de26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:05.532957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0a728ab9-cceb-4d99-8422-2ee4ab97de26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:05.534862 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1042/2096] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:13:05 2022] DELETE /identity/v3/projects/0afe9b59952a4b83aa1e8c5c6b3bedad => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:13:09.864946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225b6db-e0b5-4353-91da-9cb6aba1706d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:09.865540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225b6db-e0b5-4353-91da-9cb6aba1706d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:09.865646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225b6db-e0b5-4353-91da-9cb6aba1706d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:09.885059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1225b6db-e0b5-4353-91da-9cb6aba1706d None None] MFA Rules not processed for user `be1372c20ad94145b775663a9037d55a`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:13:09.886262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1225b6db-e0b5-4353-91da-9cb6aba1706d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:09.907281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:09.959558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1055/2097] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:13:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:13:15.527191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:19.952522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=15 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:26.833299 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001b87f-6f88-43f9-b550-943d1554f35c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:26.833957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001b87f-6f88-43f9-b550-943d1554f35c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:26.833957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001b87f-6f88-43f9-b550-943d1554f35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:26.855027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-8001b87f-6f88-43f9-b550-943d1554f35c None None] MFA Rules not processed for user `5445e009ab664626a2ab54351b0a9631`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:13:26.856335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8001b87f-6f88-43f9-b550-943d1554f35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:26.921655 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1043/2098] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:13:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:13:36.914674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=25 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:44.136529 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225b6db-e0b5-4353-91da-9cb6aba1706d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:44.139223 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1225b6db-e0b5-4353-91da-9cb6aba1706d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:44.224298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225b6db-e0b5-4353-91da-9cb6aba1706d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:44.229111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024ff90-4f5b-489d-aced-0f2ac9dc2e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:44.230025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024ff90-4f5b-489d-aced-0f2ac9dc2e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:44.230312 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024ff90-4f5b-489d-aced-0f2ac9dc2e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:44.230590 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024ff90-4f5b-489d-aced-0f2ac9dc2e15 admin admin] PATH_INFO: `/v3/users/6279105a28b343bf9eb89c3ef5c74361` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:44.231895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024ff90-4f5b-489d-aced-0f2ac9dc2e15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6279105a28b343bf9eb89c3ef5c74361)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:44.233942 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024ff90-4f5b-489d-aced-0f2ac9dc2e15 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:44.294874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6024ff90-4f5b-489d-aced-0f2ac9dc2e15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6279105a28b343bf9eb89c3ef5c74361'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:44.295712 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6024ff90-4f5b-489d-aced-0f2ac9dc2e15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6279105a28b343bf9eb89c3ef5c74361'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:44.307171 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6024ff90-4f5b-489d-aced-0f2ac9dc2e15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6279105a28b343bf9eb89c3ef5c74361'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:44.322537 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6024ff90-4f5b-489d-aced-0f2ac9dc2e15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6279105a28b343bf9eb89c3ef5c74361'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:44.330810 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6024ff90-4f5b-489d-aced-0f2ac9dc2e15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6279105a28b343bf9eb89c3ef5c74361'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:44.338501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1056/2099] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:13:44 2022] DELETE /identity/v3/users/6279105a28b343bf9eb89c3ef5c74361 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:13:44.343255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba99520-bbdc-4a97-98ee-99ddaf5ac504 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:44.343797 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba99520-bbdc-4a97-98ee-99ddaf5ac504 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:44.343797 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba99520-bbdc-4a97-98ee-99ddaf5ac504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:44.363983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-dba99520-bbdc-4a97-98ee-99ddaf5ac504 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:13:44.365290 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dba99520-bbdc-4a97-98ee-99ddaf5ac504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:44.469425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1044/2100] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:13:44.480769 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024ff90-4f5b-489d-aced-0f2ac9dc2e15 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:44.482360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6024ff90-4f5b-489d-aced-0f2ac9dc2e15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:44.578087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024ff90-4f5b-489d-aced-0f2ac9dc2e15 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:44.585048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350538e-0991-4e3a-ad85-d0575a274b8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:44.586127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350538e-0991-4e3a-ad85-d0575a274b8d service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:44.586631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350538e-0991-4e3a-ad85-d0575a274b8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:44.587147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350538e-0991-4e3a-ad85-d0575a274b8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:44.605522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350538e-0991-4e3a-ad85-d0575a274b8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:44.608510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350538e-0991-4e3a-ad85-d0575a274b8d service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:44.629821 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1057/2101] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:13:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:13:45.123849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba99520-bbdc-4a97-98ee-99ddaf5ac504 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:45.125671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dba99520-bbdc-4a97-98ee-99ddaf5ac504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:45.164482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba99520-bbdc-4a97-98ee-99ddaf5ac504 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:45.179648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3b7f8-138c-43d7-b21f-d7568a22b845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:45.180698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3b7f8-138c-43d7-b21f-d7568a22b845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:45.181053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3b7f8-138c-43d7-b21f-d7568a22b845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:45.181409 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3b7f8-138c-43d7-b21f-d7568a22b845 admin admin] PATH_INFO: `/v3/projects/3956d6c47b0e464fbbc4b30bd78326ef` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:45.185860 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3b7f8-138c-43d7-b21f-d7568a22b845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3956d6c47b0e464fbbc4b30bd78326ef)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:45.189491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3b7f8-138c-43d7-b21f-d7568a22b845 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:45.297710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-68e3b7f8-138c-43d7-b21f-d7568a22b845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3956d6c47b0e464fbbc4b30bd78326ef', 'request_id': 'req-68e3b7f8-138c-43d7-b21f-d7568a22b845'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:45.307749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-68e3b7f8-138c-43d7-b21f-d7568a22b845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3956d6c47b0e464fbbc4b30bd78326ef', 'request_id': 'req-68e3b7f8-138c-43d7-b21f-d7568a22b845'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:45.325348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-68e3b7f8-138c-43d7-b21f-d7568a22b845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3956d6c47b0e464fbbc4b30bd78326ef', 'request_id': 'req-68e3b7f8-138c-43d7-b21f-d7568a22b845'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:45.341605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-68e3b7f8-138c-43d7-b21f-d7568a22b845 admin admin] The token cache is being invalidate because project 3956d6c47b0e464fbbc4b30bd78326ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:13:45.342239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-68e3b7f8-138c-43d7-b21f-d7568a22b845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:45.343222 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-68e3b7f8-138c-43d7-b21f-d7568a22b845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:45.349277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1045/2102] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:13:45 2022] DELETE /identity/v3/projects/3956d6c47b0e464fbbc4b30bd78326ef => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:13:45.351104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350538e-0991-4e3a-ad85-d0575a274b8d service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:45.368456 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350538e-0991-4e3a-ad85-d0575a274b8d service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:45.400100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:45.533388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705c104-22ea-4d4c-823b-670297df3674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:45.535731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705c104-22ea-4d4c-823b-670297df3674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:45.536296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705c104-22ea-4d4c-823b-670297df3674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:45.537031 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705c104-22ea-4d4c-823b-670297df3674 admin admin] PATH_INFO: `/v3/users/924ff3e52ea0423eaa1f036750ae777e` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:45.539423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705c104-22ea-4d4c-823b-670297df3674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=924ff3e52ea0423eaa1f036750ae777e)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:45.544223 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705c104-22ea-4d4c-823b-670297df3674 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:45.626404 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9705c104-22ea-4d4c-823b-670297df3674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '924ff3e52ea0423eaa1f036750ae777e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:45.627106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9705c104-22ea-4d4c-823b-670297df3674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '924ff3e52ea0423eaa1f036750ae777e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:45.646149 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9705c104-22ea-4d4c-823b-670297df3674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '924ff3e52ea0423eaa1f036750ae777e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:45.677430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9705c104-22ea-4d4c-823b-670297df3674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '924ff3e52ea0423eaa1f036750ae777e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:45.691477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9705c104-22ea-4d4c-823b-670297df3674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '924ff3e52ea0423eaa1f036750ae777e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:45.706265 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1058/2103] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:13:45 2022] DELETE /identity/v3/users/924ff3e52ea0423eaa1f036750ae777e => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:13:46.153828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3b7f8-138c-43d7-b21f-d7568a22b845 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:46.156201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-68e3b7f8-138c-43d7-b21f-d7568a22b845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:46.182814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3b7f8-138c-43d7-b21f-d7568a22b845 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:46.292669 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5722f-ff3d-409f-b161-b37fb6e802f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:46.294430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5722f-ff3d-409f-b161-b37fb6e802f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:46.295021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5722f-ff3d-409f-b161-b37fb6e802f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:46.296084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5722f-ff3d-409f-b161-b37fb6e802f1 admin admin] PATH_INFO: `/v3/projects/16ae59b0600745c3838472e96815df05` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:46.300157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5722f-ff3d-409f-b161-b37fb6e802f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16ae59b0600745c3838472e96815df05)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:46.304810 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb5722f-ff3d-409f-b161-b37fb6e802f1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:46.367546 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:46.391844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8eb5722f-ff3d-409f-b161-b37fb6e802f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16ae59b0600745c3838472e96815df05', 'request_id': 'req-8eb5722f-ff3d-409f-b161-b37fb6e802f1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:46.401486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8eb5722f-ff3d-409f-b161-b37fb6e802f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16ae59b0600745c3838472e96815df05', 'request_id': 'req-8eb5722f-ff3d-409f-b161-b37fb6e802f1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:46.419817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8eb5722f-ff3d-409f-b161-b37fb6e802f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16ae59b0600745c3838472e96815df05', 'request_id': 'req-8eb5722f-ff3d-409f-b161-b37fb6e802f1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:46.436735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8eb5722f-ff3d-409f-b161-b37fb6e802f1 admin admin] The token cache is being invalidate because project 16ae59b0600745c3838472e96815df05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:13:46.437071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8eb5722f-ff3d-409f-b161-b37fb6e802f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:46.437796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8eb5722f-ff3d-409f-b161-b37fb6e802f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:46.439477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1046/2104] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:13:46 2022] DELETE /identity/v3/projects/16ae59b0600745c3838472e96815df05 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:13:46.446965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65720b-f9bd-42a1-9e82-d5d890e76f05 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:46.447771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65720b-f9bd-42a1-9e82-d5d890e76f05 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:46.447965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65720b-f9bd-42a1-9e82-d5d890e76f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:46.474029 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-5b65720b-f9bd-42a1-9e82-d5d890e76f05 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:13:46.476372 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5b65720b-f9bd-42a1-9e82-d5d890e76f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:46.568456 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1059/2105] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:13:46.576528 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a6a0b-95eb-4b74-a5d2-6a307c9eb14c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:46.576813 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a6a0b-95eb-4b74-a5d2-6a307c9eb14c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:46.577076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a6a0b-95eb-4b74-a5d2-6a307c9eb14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:46.598585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ec7a6a0b-95eb-4b74-a5d2-6a307c9eb14c None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:13:46.600010 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ec7a6a0b-95eb-4b74-a5d2-6a307c9eb14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:46.610100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1047/2106] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:13:46.619866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65720b-f9bd-42a1-9e82-d5d890e76f05 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:46.641778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65720b-f9bd-42a1-9e82-d5d890e76f05 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:46.647411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a8118-2e39-408b-8353-12cc7fbcece3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:46.648296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a8118-2e39-408b-8353-12cc7fbcece3 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:46.648583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a8118-2e39-408b-8353-12cc7fbcece3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:46.648865 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a8118-2e39-408b-8353-12cc7fbcece3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:46.649635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a8118-2e39-408b-8353-12cc7fbcece3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:13:46.649936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a8118-2e39-408b-8353-12cc7fbcece3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:46.651839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a8118-2e39-408b-8353-12cc7fbcece3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:46.666433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1060/2107] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:13:46.671245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a6a0b-95eb-4b74-a5d2-6a307c9eb14c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:46.683296 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a6a0b-95eb-4b74-a5d2-6a307c9eb14c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:46.691761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c4e2c-1d77-46ad-8fad-9735e9060e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:46.692942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c4e2c-1d77-46ad-8fad-9735e9060e4c admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:46.693706 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c4e2c-1d77-46ad-8fad-9735e9060e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:46.694192 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c4e2c-1d77-46ad-8fad-9735e9060e4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:46.695569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c4e2c-1d77-46ad-8fad-9735e9060e4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:46.699382 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c4e2c-1d77-46ad-8fad-9735e9060e4c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:46.751645 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1048/2108] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:13:46 2022] POST /identity/v3/projects => generated 389 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:13:46.753467 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a8118-2e39-408b-8353-12cc7fbcece3 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:46.771423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a8118-2e39-408b-8353-12cc7fbcece3 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:46.817772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:46.875181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb323d-76f9-469f-b74b-87eed43185cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:46.876364 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb323d-76f9-469f-b74b-87eed43185cd admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:46.876869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb323d-76f9-469f-b74b-87eed43185cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:46.877340 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb323d-76f9-469f-b74b-87eed43185cd admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:46.879209 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb323d-76f9-469f-b74b-87eed43185cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:46.882765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb323d-76f9-469f-b74b-87eed43185cd admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:46.909992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1061/2109] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:13:46 2022] POST /identity/v3/users => generated 295 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:13:46.914585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c4e2c-1d77-46ad-8fad-9735e9060e4c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:46.927432 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c4e2c-1d77-46ad-8fad-9735e9060e4c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:46.932445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e644fd-7515-4e62-8cab-82861f7e5d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:46.933142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e644fd-7515-4e62-8cab-82861f7e5d54 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:46.933427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e644fd-7515-4e62-8cab-82861f7e5d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:46.933703 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e644fd-7515-4e62-8cab-82861f7e5d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:46.934448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e644fd-7515-4e62-8cab-82861f7e5d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:46.936306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e644fd-7515-4e62-8cab-82861f7e5d54 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:46.949657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1049/2110] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:13:46.953870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb323d-76f9-469f-b74b-87eed43185cd admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:46.965833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb323d-76f9-469f-b74b-87eed43185cd admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:46.971592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3215bbd-d07c-4200-8c36-56f963006040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:46.972383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3215bbd-d07c-4200-8c36-56f963006040 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:46.972718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3215bbd-d07c-4200-8c36-56f963006040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:46.973099 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3215bbd-d07c-4200-8c36-56f963006040 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:46.974075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3215bbd-d07c-4200-8c36-56f963006040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:46.975941 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3215bbd-d07c-4200-8c36-56f963006040 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:46.988797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1062/2111] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:13:46.993448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e644fd-7515-4e62-8cab-82861f7e5d54 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:47.004881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e644fd-7515-4e62-8cab-82861f7e5d54 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:47.009789 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a2d41-a425-45c6-9567-484e8e823fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:47.010707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a2d41-a425-45c6-9567-484e8e823fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:47.010990 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a2d41-a425-45c6-9567-484e8e823fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:47.011270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a2d41-a425-45c6-9567-484e8e823fc4 admin admin] PATH_INFO: `/v3/projects/3ad5160538b94be1a3dd34632843cb9c/users/0c4f22511e5c43199ba911f8cdf2d65a/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:47.032221 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a2d41-a425-45c6-9567-484e8e823fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ad5160538b94be1a3dd34632843cb9c, user_id=0c4f22511e5c43199ba911f8cdf2d65a, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:47.036751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a2d41-a425-45c6-9567-484e8e823fc4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:47.058823 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1050/2112] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:13:46 2022] PUT /identity/v3/projects/3ad5160538b94be1a3dd34632843cb9c/users/0c4f22511e5c43199ba911f8cdf2d65a/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:13:47.073012 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be91a4-5d0d-47d7-9ca2-269be99c4749 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:47.073763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be91a4-5d0d-47d7-9ca2-269be99c4749 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:47.074275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be91a4-5d0d-47d7-9ca2-269be99c4749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:47.110814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-11be91a4-5d0d-47d7-9ca2-269be99c4749 None None] MFA Rules not processed for user `0c4f22511e5c43199ba911f8cdf2d65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:13:47.112763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-11be91a4-5d0d-47d7-9ca2-269be99c4749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:47.210997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1063/2113] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:13:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:13:47.215976 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a2d41-a425-45c6-9567-484e8e823fc4 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:47.295013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a2d41-a425-45c6-9567-484e8e823fc4 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:47.302564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509eecc-b6f6-45da-92b8-5dab7f7bb175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:47.303571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509eecc-b6f6-45da-92b8-5dab7f7bb175 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:47.304113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509eecc-b6f6-45da-92b8-5dab7f7bb175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:47.304570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509eecc-b6f6-45da-92b8-5dab7f7bb175 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:47.305834 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509eecc-b6f6-45da-92b8-5dab7f7bb175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:47.309199 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509eecc-b6f6-45da-92b8-5dab7f7bb175 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:47.356175 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1051/2114] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:13:47 2022] POST /identity/v3/projects => generated 389 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:13:47.358433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be91a4-5d0d-47d7-9ca2-269be99c4749 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:47.379048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be91a4-5d0d-47d7-9ca2-269be99c4749 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:47.476361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e90b5e-0741-4853-8da5-245ad6eca225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:47.477074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e90b5e-0741-4853-8da5-245ad6eca225 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:47.477248 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e90b5e-0741-4853-8da5-245ad6eca225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:47.477529 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e90b5e-0741-4853-8da5-245ad6eca225 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:47.478263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e90b5e-0741-4853-8da5-245ad6eca225 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:47.480395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e90b5e-0741-4853-8da5-245ad6eca225 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:47.513966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1064/2115] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:13:47 2022] POST /identity/v3/users => generated 301 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:13:47.519027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509eecc-b6f6-45da-92b8-5dab7f7bb175 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:47.533827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509eecc-b6f6-45da-92b8-5dab7f7bb175 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:47.539457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b51fd-b58a-4233-ae1d-af391c69bbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:47.540193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b51fd-b58a-4233-ae1d-af391c69bbd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:47.540487 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b51fd-b58a-4233-ae1d-af391c69bbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:47.540760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b51fd-b58a-4233-ae1d-af391c69bbd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:47.541509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b51fd-b58a-4233-ae1d-af391c69bbd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:47.543258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b51fd-b58a-4233-ae1d-af391c69bbd5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:47.566086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1052/2116] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:13:47.571704 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e90b5e-0741-4853-8da5-245ad6eca225 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:47.585576 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e90b5e-0741-4853-8da5-245ad6eca225 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:47.590879 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826ea04-2461-4f69-9621-86892456bee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:47.591809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826ea04-2461-4f69-9621-86892456bee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:47.592125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826ea04-2461-4f69-9621-86892456bee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:47.592407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826ea04-2461-4f69-9621-86892456bee7 admin admin] PATH_INFO: `/v3/projects/6da63b864f1e412c993d6a5255d012b6/users/a91a9ce5a8ad47fdb1d4c4ebb425723e/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:47.617160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826ea04-2461-4f69-9621-86892456bee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6da63b864f1e412c993d6a5255d012b6, user_id=a91a9ce5a8ad47fdb1d4c4ebb425723e, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:47.620051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826ea04-2461-4f69-9621-86892456bee7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:47.637380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1065/2117] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:13:47 2022] PUT /identity/v3/projects/6da63b864f1e412c993d6a5255d012b6/users/a91a9ce5a8ad47fdb1d4c4ebb425723e/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:13:47.642182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b51fd-b58a-4233-ae1d-af391c69bbd5 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:47.657690 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b51fd-b58a-4233-ae1d-af391c69bbd5 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:47.691692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:47.770035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fa095-c773-42e8-b8af-1935770b2724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:47.770790 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fa095-c773-42e8-b8af-1935770b2724 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:47.771169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fa095-c773-42e8-b8af-1935770b2724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:47.771565 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fa095-c773-42e8-b8af-1935770b2724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:47.772489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fa095-c773-42e8-b8af-1935770b2724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:47.774508 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fa095-c773-42e8-b8af-1935770b2724 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:47.787107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1053/2118] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:13:47.791861 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826ea04-2461-4f69-9621-86892456bee7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:47.798432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:47.807701 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826ea04-2461-4f69-9621-86892456bee7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:47.814552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446136c-3729-4dc4-a2bc-1c15eaf78d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:47.815990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446136c-3729-4dc4-a2bc-1c15eaf78d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:47.816467 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446136c-3729-4dc4-a2bc-1c15eaf78d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:47.817114 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446136c-3729-4dc4-a2bc-1c15eaf78d41 admin admin] PATH_INFO: `/v3/projects/6da63b864f1e412c993d6a5255d012b6/users/a91a9ce5a8ad47fdb1d4c4ebb425723e/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:47.819856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446136c-3729-4dc4-a2bc-1c15eaf78d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6da63b864f1e412c993d6a5255d012b6, user_id=a91a9ce5a8ad47fdb1d4c4ebb425723e, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:47.823151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f446136c-3729-4dc4-a2bc-1c15eaf78d41 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:47.838090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1066/2119] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:13:47 2022] PUT /identity/v3/projects/6da63b864f1e412c993d6a5255d012b6/users/a91a9ce5a8ad47fdb1d4c4ebb425723e/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:13:47.852975 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f8d03-7e21-45f8-b335-d813bbb08126 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:47.853539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f8d03-7e21-45f8-b335-d813bbb08126 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:47.853965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f8d03-7e21-45f8-b335-d813bbb08126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:47.903228 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f02f8d03-7e21-45f8-b335-d813bbb08126 None None] MFA Rules not processed for user `a91a9ce5a8ad47fdb1d4c4ebb425723e`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:13:47.904753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f02f8d03-7e21-45f8-b335-d813bbb08126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:48.025921 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1054/2120] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:13:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:13:48.043093 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446136c-3729-4dc4-a2bc-1c15eaf78d41 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:48.045525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f446136c-3729-4dc4-a2bc-1c15eaf78d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:48.135846 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446136c-3729-4dc4-a2bc-1c15eaf78d41 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:48.139744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b71c3-6723-4093-8a8f-75abb28e37e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:48.140457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b71c3-6723-4093-8a8f-75abb28e37e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:48.140745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b71c3-6723-4093-8a8f-75abb28e37e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:48.141022 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b71c3-6723-4093-8a8f-75abb28e37e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:48.153259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b71c3-6723-4093-8a8f-75abb28e37e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:48.154751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b71c3-6723-4093-8a8f-75abb28e37e9 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:48.167186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1067/2121] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:13:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:13:48.321798 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f8d03-7e21-45f8-b335-d813bbb08126 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:48.403768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f8d03-7e21-45f8-b335-d813bbb08126 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:48.409735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad642b4d-b4d3-4af9-95e6-f8a4ce884b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:48.411026 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad642b4d-b4d3-4af9-95e6-f8a4ce884b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:48.411330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad642b4d-b4d3-4af9-95e6-f8a4ce884b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:48.411610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad642b4d-b4d3-4af9-95e6-f8a4ce884b48 admin admin] PATH_INFO: `/v3/users/0c4f22511e5c43199ba911f8cdf2d65a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:48.413007 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad642b4d-b4d3-4af9-95e6-f8a4ce884b48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c4f22511e5c43199ba911f8cdf2d65a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:48.415014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad642b4d-b4d3-4af9-95e6-f8a4ce884b48 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:48.500877 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ad642b4d-b4d3-4af9-95e6-f8a4ce884b48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c4f22511e5c43199ba911f8cdf2d65a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:48.505514 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:48.515043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ad642b4d-b4d3-4af9-95e6-f8a4ce884b48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c4f22511e5c43199ba911f8cdf2d65a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:48.525641 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ad642b4d-b4d3-4af9-95e6-f8a4ce884b48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c4f22511e5c43199ba911f8cdf2d65a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:48.544791 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ad642b4d-b4d3-4af9-95e6-f8a4ce884b48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c4f22511e5c43199ba911f8cdf2d65a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:48.545788 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ad642b4d-b4d3-4af9-95e6-f8a4ce884b48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c4f22511e5c43199ba911f8cdf2d65a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:48.554209 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1055/2122] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:13:48 2022] DELETE /identity/v3/users/0c4f22511e5c43199ba911f8cdf2d65a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:13:48.559107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f705986-b200-401d-975f-75be94a8e32f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:48.559744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f705986-b200-401d-975f-75be94a8e32f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:48.559744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f705986-b200-401d-975f-75be94a8e32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:48.594583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-2f705986-b200-401d-975f-75be94a8e32f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:13:48.597419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2f705986-b200-401d-975f-75be94a8e32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:48.701819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1068/2123] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:13:48.715092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad642b4d-b4d3-4af9-95e6-f8a4ce884b48 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:48.718027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ad642b4d-b4d3-4af9-95e6-f8a4ce884b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:48.824926 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad642b4d-b4d3-4af9-95e6-f8a4ce884b48 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:48.833131 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91035d0-2cd4-4ee3-b553-dbafef9a501a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:48.834430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91035d0-2cd4-4ee3-b553-dbafef9a501a service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:48.835103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91035d0-2cd4-4ee3-b553-dbafef9a501a service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:48.835604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91035d0-2cd4-4ee3-b553-dbafef9a501a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:48.854667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91035d0-2cd4-4ee3-b553-dbafef9a501a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:48.858039 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91035d0-2cd4-4ee3-b553-dbafef9a501a service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:48.882065 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1056/2124] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:13:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:13:49.438544 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f705986-b200-401d-975f-75be94a8e32f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:49.440183 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2f705986-b200-401d-975f-75be94a8e32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:49.455661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f705986-b200-401d-975f-75be94a8e32f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:49.462594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:49.463584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:49.464030 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:49.464439 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe admin admin] PATH_INFO: `/v3/projects/3ad5160538b94be1a3dd34632843cb9c` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:49.466094 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ad5160538b94be1a3dd34632843cb9c)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:49.468888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:49.551097 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ad5160538b94be1a3dd34632843cb9c', 'request_id': 'req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:49.560044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ad5160538b94be1a3dd34632843cb9c', 'request_id': 'req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:49.568862 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ad5160538b94be1a3dd34632843cb9c', 'request_id': 'req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:49.585163 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe admin admin] The token cache is being invalidate because project 3ad5160538b94be1a3dd34632843cb9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:13:49.585497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:49.586142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:49.587807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1069/2125] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:13:49 2022] DELETE /identity/v3/projects/3ad5160538b94be1a3dd34632843cb9c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:13:49.594047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91035d0-2cd4-4ee3-b553-dbafef9a501a service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:49.609141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91035d0-2cd4-4ee3-b553-dbafef9a501a service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:49.676270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b880a-e89d-436a-a92c-cd13b3f5ff39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:49.677162 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b880a-e89d-436a-a92c-cd13b3f5ff39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:49.677449 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b880a-e89d-436a-a92c-cd13b3f5ff39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:49.677733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b880a-e89d-436a-a92c-cd13b3f5ff39 admin admin] PATH_INFO: `/v3/users/a91a9ce5a8ad47fdb1d4c4ebb425723e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:49.679027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b880a-e89d-436a-a92c-cd13b3f5ff39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a91a9ce5a8ad47fdb1d4c4ebb425723e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:49.681085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b880a-e89d-436a-a92c-cd13b3f5ff39 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:49.745648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f33b880a-e89d-436a-a92c-cd13b3f5ff39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a91a9ce5a8ad47fdb1d4c4ebb425723e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:49.758702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f33b880a-e89d-436a-a92c-cd13b3f5ff39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a91a9ce5a8ad47fdb1d4c4ebb425723e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:49.767171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f33b880a-e89d-436a-a92c-cd13b3f5ff39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a91a9ce5a8ad47fdb1d4c4ebb425723e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:49.775195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:49.785963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f33b880a-e89d-436a-a92c-cd13b3f5ff39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a91a9ce5a8ad47fdb1d4c4ebb425723e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:49.787613 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f33b880a-e89d-436a-a92c-cd13b3f5ff39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a91a9ce5a8ad47fdb1d4c4ebb425723e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:49.799114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1057/2126] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:13:49 2022] DELETE /identity/v3/users/a91a9ce5a8ad47fdb1d4c4ebb425723e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:13:50.139025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:50.140869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:50.160019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f59bc8-8d80-48ee-aa28-1a30a9ce87fe admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:50.176914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:50.248274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a753d-fc16-4a3b-b785-156ea1ad841c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:50.249099 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a753d-fc16-4a3b-b785-156ea1ad841c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:50.249433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a753d-fc16-4a3b-b785-156ea1ad841c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:50.249782 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a753d-fc16-4a3b-b785-156ea1ad841c admin admin] PATH_INFO: `/v3/projects/6da63b864f1e412c993d6a5255d012b6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:50.251330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a753d-fc16-4a3b-b785-156ea1ad841c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6da63b864f1e412c993d6a5255d012b6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:50.254249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a753d-fc16-4a3b-b785-156ea1ad841c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:50.316563 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6e9a753d-fc16-4a3b-b785-156ea1ad841c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6da63b864f1e412c993d6a5255d012b6', 'request_id': 'req-6e9a753d-fc16-4a3b-b785-156ea1ad841c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:50.323208 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6e9a753d-fc16-4a3b-b785-156ea1ad841c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6da63b864f1e412c993d6a5255d012b6', 'request_id': 'req-6e9a753d-fc16-4a3b-b785-156ea1ad841c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:50.331184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6e9a753d-fc16-4a3b-b785-156ea1ad841c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6da63b864f1e412c993d6a5255d012b6', 'request_id': 'req-6e9a753d-fc16-4a3b-b785-156ea1ad841c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:50.347648 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6e9a753d-fc16-4a3b-b785-156ea1ad841c admin admin] The token cache is being invalidate because project 6da63b864f1e412c993d6a5255d012b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:13:50.348161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6e9a753d-fc16-4a3b-b785-156ea1ad841c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:50.348701 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6e9a753d-fc16-4a3b-b785-156ea1ad841c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:13:50.350371 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1070/2127] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:13:50 2022] DELETE /identity/v3/projects/6da63b864f1e412c993d6a5255d012b6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:13:50.356833 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8b2ae-77b0-4fdb-afd4-576a5658306b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:50.357117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8b2ae-77b0-4fdb-afd4-576a5658306b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:50.357381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8b2ae-77b0-4fdb-afd4-576a5658306b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:50.377669 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-52f8b2ae-77b0-4fdb-afd4-576a5658306b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:13:50.378919 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-52f8b2ae-77b0-4fdb-afd4-576a5658306b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:50.454243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1058/2128] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:13:50.464673 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd33f8c-ef63-438e-af1e-6de9b9e263e1 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:50.465001 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd33f8c-ef63-438e-af1e-6de9b9e263e1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:50.465226 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd33f8c-ef63-438e-af1e-6de9b9e263e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:50.485540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-afd33f8c-ef63-438e-af1e-6de9b9e263e1 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:13:50.486788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-afd33f8c-ef63-438e-af1e-6de9b9e263e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:50.496603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1071/2129] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:13:50.501407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8b2ae-77b0-4fdb-afd4-576a5658306b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:50.513946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8b2ae-77b0-4fdb-afd4-576a5658306b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:50.519822 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd78a9-32f2-44fa-b64e-190797ba1fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:50.520543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd78a9-32f2-44fa-b64e-190797ba1fae admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:50.520824 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd78a9-32f2-44fa-b64e-190797ba1fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:50.521100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd78a9-32f2-44fa-b64e-190797ba1fae admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:50.521849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd78a9-32f2-44fa-b64e-190797ba1fae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:13:50.522156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd78a9-32f2-44fa-b64e-190797ba1fae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:50.523854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd78a9-32f2-44fa-b64e-190797ba1fae admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:50.544699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1059/2130] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:13:50.551415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd33f8c-ef63-438e-af1e-6de9b9e263e1 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:50.564089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd33f8c-ef63-438e-af1e-6de9b9e263e1 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:50.571727 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b37e0-48a1-43ad-8ac6-80e36684ae3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:50.572846 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b37e0-48a1-43ad-8ac6-80e36684ae3a admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:50.573310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b37e0-48a1-43ad-8ac6-80e36684ae3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:50.573775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b37e0-48a1-43ad-8ac6-80e36684ae3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:50.574979 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b37e0-48a1-43ad-8ac6-80e36684ae3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:50.578196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b37e0-48a1-43ad-8ac6-80e36684ae3a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:50.624928 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1072/2131] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:13:50 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:13:50.638191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd78a9-32f2-44fa-b64e-190797ba1fae admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:50.661104 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd78a9-32f2-44fa-b64e-190797ba1fae admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:50.772211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba898782-07bd-479f-b681-5311725b4a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:50.773158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba898782-07bd-479f-b681-5311725b4a27 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:50.773627 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba898782-07bd-479f-b681-5311725b4a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:50.774087 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba898782-07bd-479f-b681-5311725b4a27 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:50.775289 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba898782-07bd-479f-b681-5311725b4a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:50.778934 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba898782-07bd-479f-b681-5311725b4a27 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:50.811979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1060/2132] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:13:50 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:13:50.817324 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b37e0-48a1-43ad-8ac6-80e36684ae3a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:50.828957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b37e0-48a1-43ad-8ac6-80e36684ae3a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:50.834687 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbd909-e1bc-4ec4-8168-d3318f8ac532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:50.835373 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbd909-e1bc-4ec4-8168-d3318f8ac532 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:50.835656 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbd909-e1bc-4ec4-8168-d3318f8ac532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:50.835964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbd909-e1bc-4ec4-8168-d3318f8ac532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:50.836716 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbd909-e1bc-4ec4-8168-d3318f8ac532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:50.838375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbd909-e1bc-4ec4-8168-d3318f8ac532 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:50.851657 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1073/2133] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:13:50.856100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba898782-07bd-479f-b681-5311725b4a27 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:50.869017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba898782-07bd-479f-b681-5311725b4a27 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:50.874689 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6b00f-066f-4ade-9e8a-e3d87a086f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:50.875354 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6b00f-066f-4ade-9e8a-e3d87a086f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:50.875647 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6b00f-066f-4ade-9e8a-e3d87a086f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:50.876028 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6b00f-066f-4ade-9e8a-e3d87a086f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:50.876784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6b00f-066f-4ade-9e8a-e3d87a086f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:50.879207 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6b00f-066f-4ade-9e8a-e3d87a086f54 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:50.891246 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1061/2134] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:13:50.896098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbd909-e1bc-4ec4-8168-d3318f8ac532 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:50.907344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbd909-e1bc-4ec4-8168-d3318f8ac532 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:50.913505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44139c3f-9269-4452-ab36-cfa199301e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:50.914418 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44139c3f-9269-4452-ab36-cfa199301e26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:50.914694 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44139c3f-9269-4452-ab36-cfa199301e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:50.914971 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44139c3f-9269-4452-ab36-cfa199301e26 admin admin] PATH_INFO: `/v3/projects/f6564a02f1d0478695f20f6bf930dd97/users/a22dc68c4ce548b1b70a9f0aaceb9ea1/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:50.938118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44139c3f-9269-4452-ab36-cfa199301e26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6564a02f1d0478695f20f6bf930dd97, user_id=a22dc68c4ce548b1b70a9f0aaceb9ea1, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:50.943723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44139c3f-9269-4452-ab36-cfa199301e26 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:50.967269 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1074/2135] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:13:50 2022] PUT /identity/v3/projects/f6564a02f1d0478695f20f6bf930dd97/users/a22dc68c4ce548b1b70a9f0aaceb9ea1/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:13:50.980143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad584bc-7508-4adf-8701-3abc274a9dd2 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:50.980518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad584bc-7508-4adf-8701-3abc274a9dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:50.980866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad584bc-7508-4adf-8701-3abc274a9dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:51.015121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-cad584bc-7508-4adf-8701-3abc274a9dd2 None None] MFA Rules not processed for user `a22dc68c4ce548b1b70a9f0aaceb9ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:13:51.016310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-cad584bc-7508-4adf-8701-3abc274a9dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:51.069704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:51.100525 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1062/2136] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:13:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:13:51.105564 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44139c3f-9269-4452-ab36-cfa199301e26 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:51.159225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:51.182814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44139c3f-9269-4452-ab36-cfa199301e26 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:51.187849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88620bf-55e1-447b-8ddd-10335786b054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:51.188569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88620bf-55e1-447b-8ddd-10335786b054 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:51.188855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88620bf-55e1-447b-8ddd-10335786b054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:51.189136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88620bf-55e1-447b-8ddd-10335786b054 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:51.189848 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88620bf-55e1-447b-8ddd-10335786b054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:51.192894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88620bf-55e1-447b-8ddd-10335786b054 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:51.224637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1075/2137] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:13:51 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:13:51.228212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad584bc-7508-4adf-8701-3abc274a9dd2 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:51.240869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad584bc-7508-4adf-8701-3abc274a9dd2 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:51.310147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aeaf02-58ce-40ab-8932-eb1cc71a6baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:51.310741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aeaf02-58ce-40ab-8932-eb1cc71a6baa admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:51.311031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aeaf02-58ce-40ab-8932-eb1cc71a6baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:51.311310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aeaf02-58ce-40ab-8932-eb1cc71a6baa admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:51.312082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aeaf02-58ce-40ab-8932-eb1cc71a6baa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:51.314137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aeaf02-58ce-40ab-8932-eb1cc71a6baa admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:51.338524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1063/2138] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:13:51 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:13:51.343110 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88620bf-55e1-447b-8ddd-10335786b054 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:51.361732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88620bf-55e1-447b-8ddd-10335786b054 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:51.371552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9da31-a7e9-4e26-a687-95df35274036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:51.372804 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9da31-a7e9-4e26-a687-95df35274036 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:51.373302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9da31-a7e9-4e26-a687-95df35274036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:51.373802 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9da31-a7e9-4e26-a687-95df35274036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:51.375227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9da31-a7e9-4e26-a687-95df35274036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:51.378720 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f9da31-a7e9-4e26-a687-95df35274036 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:51.398460 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1076/2139] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:13:51.403072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aeaf02-58ce-40ab-8932-eb1cc71a6baa admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:51.415512 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aeaf02-58ce-40ab-8932-eb1cc71a6baa admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:51.421254 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaf7f1-c9c0-4101-a374-a2f592497fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:51.422088 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaf7f1-c9c0-4101-a374-a2f592497fce admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:51.422516 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaf7f1-c9c0-4101-a374-a2f592497fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:51.422899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaf7f1-c9c0-4101-a374-a2f592497fce admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:51.423761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecaf7f1-c9c0-4101-a374-a2f592497fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:51.426679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecaf7f1-c9c0-4101-a374-a2f592497fce admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:51.443044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1064/2140] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:13:51.447541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9da31-a7e9-4e26-a687-95df35274036 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:51.460192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9da31-a7e9-4e26-a687-95df35274036 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:51.465219 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e36ba-0cc8-4bea-af7d-b698401388a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:51.466169 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e36ba-0cc8-4bea-af7d-b698401388a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:51.466453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e36ba-0cc8-4bea-af7d-b698401388a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:51.466731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e36ba-0cc8-4bea-af7d-b698401388a6 admin admin] PATH_INFO: `/v3/projects/d957b5f22b2943dd9e58a8ca4013525e/users/88299b1153004c71a8c463d596053f71/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:51.488389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e36ba-0cc8-4bea-af7d-b698401388a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d957b5f22b2943dd9e58a8ca4013525e, user_id=88299b1153004c71a8c463d596053f71, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:51.492071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e36ba-0cc8-4bea-af7d-b698401388a6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:51.510265 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1077/2141] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:13:51 2022] PUT /identity/v3/projects/d957b5f22b2943dd9e58a8ca4013525e/users/88299b1153004c71a8c463d596053f71/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:13:51.522647 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27303386-01fe-4112-b661-b949f49f12f8 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:51.523054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27303386-01fe-4112-b661-b949f49f12f8 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:51.523434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27303386-01fe-4112-b661-b949f49f12f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:51.558361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-27303386-01fe-4112-b661-b949f49f12f8 None None] MFA Rules not processed for user `88299b1153004c71a8c463d596053f71`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:13:51.559674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-27303386-01fe-4112-b661-b949f49f12f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:51.639390 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1065/2142] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:13:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:13:51.645009 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e36ba-0cc8-4bea-af7d-b698401388a6 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:51.723177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e36ba-0cc8-4bea-af7d-b698401388a6 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:51.728873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd66ccd-528f-4c7a-8dff-0d51fcf8b63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:51.729556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd66ccd-528f-4c7a-8dff-0d51fcf8b63a admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:51.729843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd66ccd-528f-4c7a-8dff-0d51fcf8b63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:51.730124 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd66ccd-528f-4c7a-8dff-0d51fcf8b63a admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:51.730840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd66ccd-528f-4c7a-8dff-0d51fcf8b63a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:51.732878 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd66ccd-528f-4c7a-8dff-0d51fcf8b63a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:51.775526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1078/2143] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:13:51 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:13:51.781439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27303386-01fe-4112-b661-b949f49f12f8 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:51.795178 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27303386-01fe-4112-b661-b949f49f12f8 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:51.863388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:51.876536 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b2e13-002d-4893-abbc-03fc5ea1e67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:51.877329 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b2e13-002d-4893-abbc-03fc5ea1e67b admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:51.878026 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b2e13-002d-4893-abbc-03fc5ea1e67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:51.878026 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b2e13-002d-4893-abbc-03fc5ea1e67b admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:51.878641 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b2e13-002d-4893-abbc-03fc5ea1e67b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:51.880720 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b2e13-002d-4893-abbc-03fc5ea1e67b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:51.914852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1066/2144] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:13:51 2022] POST /identity/v3/users => generated 299 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:13:51.919969 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd66ccd-528f-4c7a-8dff-0d51fcf8b63a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:51.935208 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd66ccd-528f-4c7a-8dff-0d51fcf8b63a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:51.943057 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd649157-71c5-400a-a61d-908fe8b3ee47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:51.944041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd649157-71c5-400a-a61d-908fe8b3ee47 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:51.944459 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd649157-71c5-400a-a61d-908fe8b3ee47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:51.944888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd649157-71c5-400a-a61d-908fe8b3ee47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:51.945800 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd649157-71c5-400a-a61d-908fe8b3ee47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:51.947870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd649157-71c5-400a-a61d-908fe8b3ee47 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:51.965979 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1079/2145] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:13:51.973126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b2e13-002d-4893-abbc-03fc5ea1e67b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:51.988454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b2e13-002d-4893-abbc-03fc5ea1e67b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:51.999643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8262e-0308-4c1f-9f8c-1d7ca4319d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:52.001318 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8262e-0308-4c1f-9f8c-1d7ca4319d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:52.001879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8262e-0308-4c1f-9f8c-1d7ca4319d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:52.002377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8262e-0308-4c1f-9f8c-1d7ca4319d76 admin admin] PATH_INFO: `/v3/projects/24afb861db7d43abbd7164b256f408d1/users/b864e5351c9641ca923d5d938239cc04/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:52.035452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8262e-0308-4c1f-9f8c-1d7ca4319d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24afb861db7d43abbd7164b256f408d1, user_id=b864e5351c9641ca923d5d938239cc04, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:52.039487 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8262e-0308-4c1f-9f8c-1d7ca4319d76 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:52.060788 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1067/2146] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:13:51 2022] PUT /identity/v3/projects/24afb861db7d43abbd7164b256f408d1/users/b864e5351c9641ca923d5d938239cc04/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:13:52.067372 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd649157-71c5-400a-a61d-908fe8b3ee47 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:52.081178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd649157-71c5-400a-a61d-908fe8b3ee47 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:52.150912 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2724679-3ba8-472f-92c5-f6c451e95e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:52.151784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2724679-3ba8-472f-92c5-f6c451e95e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:52.152167 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2724679-3ba8-472f-92c5-f6c451e95e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:52.152505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2724679-3ba8-472f-92c5-f6c451e95e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:52.153290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2724679-3ba8-472f-92c5-f6c451e95e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:52.155147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2724679-3ba8-472f-92c5-f6c451e95e1d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:52.166636 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1080/2147] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:13:52.171452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8262e-0308-4c1f-9f8c-1d7ca4319d76 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:52.184394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8262e-0308-4c1f-9f8c-1d7ca4319d76 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:52.189805 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00d673-3d93-4422-8817-7a710ab33ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:52.192019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00d673-3d93-4422-8817-7a710ab33ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:52.192406 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00d673-3d93-4422-8817-7a710ab33ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:52.192876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00d673-3d93-4422-8817-7a710ab33ad1 admin admin] PATH_INFO: `/v3/projects/24afb861db7d43abbd7164b256f408d1/users/b864e5351c9641ca923d5d938239cc04/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:52.195396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00d673-3d93-4422-8817-7a710ab33ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24afb861db7d43abbd7164b256f408d1, user_id=b864e5351c9641ca923d5d938239cc04, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:52.198499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00d673-3d93-4422-8817-7a710ab33ad1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:52.212020 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1068/2148] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:13:52 2022] PUT /identity/v3/projects/24afb861db7d43abbd7164b256f408d1/users/b864e5351c9641ca923d5d938239cc04/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:13:52.224860 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da2568-472d-4599-93f2-6bc96432662a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:52.225296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da2568-472d-4599-93f2-6bc96432662a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:52.225683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da2568-472d-4599-93f2-6bc96432662a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:52.232457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:13:52.266218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e9da2568-472d-4599-93f2-6bc96432662a None None] MFA Rules not processed for user `b864e5351c9641ca923d5d938239cc04`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:13:52.267487 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e9da2568-472d-4599-93f2-6bc96432662a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:52.364636 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1081/2149] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:13:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:13:52.378346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00d673-3d93-4422-8817-7a710ab33ad1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:52.380321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-db00d673-3d93-4422-8817-7a710ab33ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:52.463393 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00d673-3d93-4422-8817-7a710ab33ad1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:52.467157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee8b5e-7d99-494d-9537-69d96bbfc99e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:52.467871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee8b5e-7d99-494d-9537-69d96bbfc99e service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:52.468186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee8b5e-7d99-494d-9537-69d96bbfc99e service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:52.468458 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee8b5e-7d99-494d-9537-69d96bbfc99e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:52.553885 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee8b5e-7d99-494d-9537-69d96bbfc99e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:52.555486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee8b5e-7d99-494d-9537-69d96bbfc99e service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:52.565970 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1069/2150] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:13:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:13:55.294204 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da2568-472d-4599-93f2-6bc96432662a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:55.296252 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e9da2568-472d-4599-93f2-6bc96432662a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:13:55.316828 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da2568-472d-4599-93f2-6bc96432662a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:55.321338 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae4f1a-76e7-4ab1-bdd5-b9f3968da21c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:55.322190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae4f1a-76e7-4ab1-bdd5-b9f3968da21c service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:55.322474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae4f1a-76e7-4ab1-bdd5-b9f3968da21c service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:55.322749 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae4f1a-76e7-4ab1-bdd5-b9f3968da21c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:55.408092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae4f1a-76e7-4ab1-bdd5-b9f3968da21c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:55.410250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae4f1a-76e7-4ab1-bdd5-b9f3968da21c service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:55.421371 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1082/2151] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:13:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:13:56.746465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee8b5e-7d99-494d-9537-69d96bbfc99e service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:13:56.761244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee8b5e-7d99-494d-9537-69d96bbfc99e service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:13:56.765232 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d966c7-9135-43f9-ad91-d20cd27fab84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:13:56.765935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d966c7-9135-43f9-ad91-d20cd27fab84 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:13:56.766221 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d966c7-9135-43f9-ad91-d20cd27fab84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:13:56.766498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d966c7-9135-43f9-ad91-d20cd27fab84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:13:56.777560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d966c7-9135-43f9-ad91-d20cd27fab84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:13:56.779236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d966c7-9135-43f9-ad91-d20cd27fab84 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:13:56.791108 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1070/2152] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:13:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:14:05.416574 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=61 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:14:06.786270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=75,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:14:09.445735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae4f1a-76e7-4ab1-bdd5-b9f3968da21c service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:09.525616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae4f1a-76e7-4ab1-bdd5-b9f3968da21c service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:09.531940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159781af-fe6e-4045-87d5-73357dcb280a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:09.533277 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159781af-fe6e-4045-87d5-73357dcb280a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:09.533772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159781af-fe6e-4045-87d5-73357dcb280a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:09.534422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159781af-fe6e-4045-87d5-73357dcb280a admin admin] PATH_INFO: `/v3/users/a22dc68c4ce548b1b70a9f0aaceb9ea1` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:09.536507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159781af-fe6e-4045-87d5-73357dcb280a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a22dc68c4ce548b1b70a9f0aaceb9ea1)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:09.540239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159781af-fe6e-4045-87d5-73357dcb280a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:09.576759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d966c7-9135-43f9-ad91-d20cd27fab84 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:09.579199 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-55d966c7-9135-43f9-ad91-d20cd27fab84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:09.617321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-159781af-fe6e-4045-87d5-73357dcb280a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a22dc68c4ce548b1b70a9f0aaceb9ea1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:09.618036 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-159781af-fe6e-4045-87d5-73357dcb280a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a22dc68c4ce548b1b70a9f0aaceb9ea1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:09.628759 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-159781af-fe6e-4045-87d5-73357dcb280a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a22dc68c4ce548b1b70a9f0aaceb9ea1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:09.646330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-159781af-fe6e-4045-87d5-73357dcb280a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a22dc68c4ce548b1b70a9f0aaceb9ea1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:09.654866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-159781af-fe6e-4045-87d5-73357dcb280a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a22dc68c4ce548b1b70a9f0aaceb9ea1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:09.663054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d966c7-9135-43f9-ad91-d20cd27fab84 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:09.667068 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e107f0-618d-4d29-9950-b1cf0b27c686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:09.668177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e107f0-618d-4d29-9950-b1cf0b27c686 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:09.668562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e107f0-618d-4d29-9950-b1cf0b27c686 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:09.669022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e107f0-618d-4d29-9950-b1cf0b27c686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:09.671381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b8e107f0-618d-4d29-9950-b1cf0b27c686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:09.678529 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1083/2153] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:14:09 2022] DELETE /identity/v3/users/a22dc68c4ce548b1b70a9f0aaceb9ea1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:14:09.685827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e55a0-2684-4e42-aeb0-aeef50d73e0e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:09.686324 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e55a0-2684-4e42-aeb0-aeef50d73e0e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:09.686817 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e55a0-2684-4e42-aeb0-aeef50d73e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:09.716398 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-835e55a0-2684-4e42-aeb0-aeef50d73e0e None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:14:09.717661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-835e55a0-2684-4e42-aeb0-aeef50d73e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:09.769385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e107f0-618d-4d29-9950-b1cf0b27c686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:09.770890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e107f0-618d-4d29-9950-b1cf0b27c686 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:09.786455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1071/2154] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:14:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:14:09.805401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1084/2155] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:14:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:14:09.816681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e107f0-618d-4d29-9950-b1cf0b27c686 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:09.818456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b8e107f0-618d-4d29-9950-b1cf0b27c686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:09.831035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e107f0-618d-4d29-9950-b1cf0b27c686 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:09.893375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9363a0b-ceeb-4af5-883e-b021c35e945a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:09.894339 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9363a0b-ceeb-4af5-883e-b021c35e945a service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:09.894676 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9363a0b-ceeb-4af5-883e-b021c35e945a service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:09.895039 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9363a0b-ceeb-4af5-883e-b021c35e945a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:09.911402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9363a0b-ceeb-4af5-883e-b021c35e945a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:09.913545 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9363a0b-ceeb-4af5-883e-b021c35e945a service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:09.928904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1072/2156] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:14:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:14:10.404283 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e55a0-2684-4e42-aeb0-aeef50d73e0e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:10.406790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-835e55a0-2684-4e42-aeb0-aeef50d73e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:10.459984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e55a0-2684-4e42-aeb0-aeef50d73e0e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:10.471200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de53cc-cddb-49c2-b29c-f36518be98e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:10.473988 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de53cc-cddb-49c2-b29c-f36518be98e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:10.475145 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de53cc-cddb-49c2-b29c-f36518be98e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:10.476984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de53cc-cddb-49c2-b29c-f36518be98e7 admin admin] PATH_INFO: `/v3/projects/f6564a02f1d0478695f20f6bf930dd97` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:10.480667 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de53cc-cddb-49c2-b29c-f36518be98e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6564a02f1d0478695f20f6bf930dd97)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:10.484572 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de53cc-cddb-49c2-b29c-f36518be98e7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:10.514965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:14:10.555630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-30de53cc-cddb-49c2-b29c-f36518be98e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6564a02f1d0478695f20f6bf930dd97', 'request_id': 'req-30de53cc-cddb-49c2-b29c-f36518be98e7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:10.563241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-30de53cc-cddb-49c2-b29c-f36518be98e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6564a02f1d0478695f20f6bf930dd97', 'request_id': 'req-30de53cc-cddb-49c2-b29c-f36518be98e7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:10.572474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-30de53cc-cddb-49c2-b29c-f36518be98e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6564a02f1d0478695f20f6bf930dd97', 'request_id': 'req-30de53cc-cddb-49c2-b29c-f36518be98e7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:10.590405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-30de53cc-cddb-49c2-b29c-f36518be98e7 admin admin] The token cache is being invalidate because project f6564a02f1d0478695f20f6bf930dd97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:14:10.590697 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-30de53cc-cddb-49c2-b29c-f36518be98e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:10.591399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-30de53cc-cddb-49c2-b29c-f36518be98e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:10.593043 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1085/2157] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:14:10 2022] DELETE /identity/v3/projects/f6564a02f1d0478695f20f6bf930dd97 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:14:10.598523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9363a0b-ceeb-4af5-883e-b021c35e945a service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:10.613411 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9363a0b-ceeb-4af5-883e-b021c35e945a service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:10.682446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:14:10.694154 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e72fce-7d48-4b34-b339-ed465d362f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:10.694975 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e72fce-7d48-4b34-b339-ed465d362f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:10.695344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e72fce-7d48-4b34-b339-ed465d362f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:10.695535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e72fce-7d48-4b34-b339-ed465d362f5e admin admin] PATH_INFO: `/v3/users/88299b1153004c71a8c463d596053f71` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:10.696869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e72fce-7d48-4b34-b339-ed465d362f5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=88299b1153004c71a8c463d596053f71)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:10.698902 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e72fce-7d48-4b34-b339-ed465d362f5e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:10.764309 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b8e72fce-7d48-4b34-b339-ed465d362f5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88299b1153004c71a8c463d596053f71'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:10.776986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b8e72fce-7d48-4b34-b339-ed465d362f5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88299b1153004c71a8c463d596053f71'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:10.794869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b8e72fce-7d48-4b34-b339-ed465d362f5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88299b1153004c71a8c463d596053f71'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:10.819207 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b8e72fce-7d48-4b34-b339-ed465d362f5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88299b1153004c71a8c463d596053f71'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:10.820394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b8e72fce-7d48-4b34-b339-ed465d362f5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88299b1153004c71a8c463d596053f71'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:10.832792 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1073/2158] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:14:10 2022] DELETE /identity/v3/users/88299b1153004c71a8c463d596053f71 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:14:11.247039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de53cc-cddb-49c2-b29c-f36518be98e7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:11.247039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-30de53cc-cddb-49c2-b29c-f36518be98e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:11.282867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de53cc-cddb-49c2-b29c-f36518be98e7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:11.396224 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:11.397695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:11.398226 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:11.398734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23 admin admin] PATH_INFO: `/v3/projects/d957b5f22b2943dd9e58a8ca4013525e` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:11.401145 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d957b5f22b2943dd9e58a8ca4013525e)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:11.405047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:11.484418 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd957b5f22b2943dd9e58a8ca4013525e', 'request_id': 'req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:11.492647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd957b5f22b2943dd9e58a8ca4013525e', 'request_id': 'req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:11.501129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd957b5f22b2943dd9e58a8ca4013525e', 'request_id': 'req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:11.520120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23 admin admin] The token cache is being invalidate because project d957b5f22b2943dd9e58a8ca4013525e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:14:11.520415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:11.521149 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:11.522790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1086/2159] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:14:11 2022] DELETE /identity/v3/projects/d957b5f22b2943dd9e58a8ca4013525e => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:14:11.527133 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e72fce-7d48-4b34-b339-ed465d362f5e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:11.528756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b8e72fce-7d48-4b34-b339-ed465d362f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:11.553415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e72fce-7d48-4b34-b339-ed465d362f5e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:11.656865 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace3ac4-96af-4e05-836f-2e1502c184df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:11.658834 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace3ac4-96af-4e05-836f-2e1502c184df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:11.659870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace3ac4-96af-4e05-836f-2e1502c184df admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:11.660372 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace3ac4-96af-4e05-836f-2e1502c184df admin admin] PATH_INFO: `/v3/users/b864e5351c9641ca923d5d938239cc04` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:11.662371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace3ac4-96af-4e05-836f-2e1502c184df admin admin] RBAC: Authorizing `identity:delete_user(user_id=b864e5351c9641ca923d5d938239cc04)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:11.664863 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace3ac4-96af-4e05-836f-2e1502c184df admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:11.739617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dace3ac4-96af-4e05-836f-2e1502c184df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b864e5351c9641ca923d5d938239cc04'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:11.750463 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dace3ac4-96af-4e05-836f-2e1502c184df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b864e5351c9641ca923d5d938239cc04'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:11.768520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dace3ac4-96af-4e05-836f-2e1502c184df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b864e5351c9641ca923d5d938239cc04'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:11.786150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:14:11.797496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dace3ac4-96af-4e05-836f-2e1502c184df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b864e5351c9641ca923d5d938239cc04'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:11.798764 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dace3ac4-96af-4e05-836f-2e1502c184df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b864e5351c9641ca923d5d938239cc04'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:11.813444 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1074/2160] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:14:11 2022] DELETE /identity/v3/users/b864e5351c9641ca923d5d938239cc04 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:14:12.178255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:12.180310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:12.202897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bfbc5-83d3-4c2e-9247-008037f9ff23 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:12.244306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:14:12.280499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aaaf3e-9693-4b3f-be2e-59b598a54f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:12.281399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaaf3e-9693-4b3f-be2e-59b598a54f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:12.281704 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaaf3e-9693-4b3f-be2e-59b598a54f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:12.281989 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaaf3e-9693-4b3f-be2e-59b598a54f68 admin admin] PATH_INFO: `/v3/projects/24afb861db7d43abbd7164b256f408d1` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:12.284597 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aaaf3e-9693-4b3f-be2e-59b598a54f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24afb861db7d43abbd7164b256f408d1)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:12.286740 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aaaf3e-9693-4b3f-be2e-59b598a54f68 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:12.355699 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-17aaaf3e-9693-4b3f-be2e-59b598a54f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24afb861db7d43abbd7164b256f408d1', 'request_id': 'req-17aaaf3e-9693-4b3f-be2e-59b598a54f68'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:12.365180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-17aaaf3e-9693-4b3f-be2e-59b598a54f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24afb861db7d43abbd7164b256f408d1', 'request_id': 'req-17aaaf3e-9693-4b3f-be2e-59b598a54f68'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:12.375300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-17aaaf3e-9693-4b3f-be2e-59b598a54f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24afb861db7d43abbd7164b256f408d1', 'request_id': 'req-17aaaf3e-9693-4b3f-be2e-59b598a54f68'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:12.396748 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-17aaaf3e-9693-4b3f-be2e-59b598a54f68 admin admin] The token cache is being invalidate because project 24afb861db7d43abbd7164b256f408d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:14:12.397156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-17aaaf3e-9693-4b3f-be2e-59b598a54f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:12.399745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-17aaaf3e-9693-4b3f-be2e-59b598a54f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:14:12.401613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1087/2161] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:14:12 2022] DELETE /identity/v3/projects/24afb861db7d43abbd7164b256f408d1 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:14:12.409084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662611c2-b422-4c39-bb82-618221db4fb9 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:12.409827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662611c2-b422-4c39-bb82-618221db4fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:12.410166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662611c2-b422-4c39-bb82-618221db4fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:12.433610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-662611c2-b422-4c39-bb82-618221db4fb9 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:14:12.434937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-662611c2-b422-4c39-bb82-618221db4fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:12.538997 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1075/2162] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:14:12.552913 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52828178-7143-4488-9ceb-478e8be840bd None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:12.553683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52828178-7143-4488-9ceb-478e8be840bd None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:12.554199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52828178-7143-4488-9ceb-478e8be840bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:12.578379 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-52828178-7143-4488-9ceb-478e8be840bd None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:14:12.579663 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-52828178-7143-4488-9ceb-478e8be840bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:12.588835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1088/2163] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:14:12.594694 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662611c2-b422-4c39-bb82-618221db4fb9 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:12.614498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662611c2-b422-4c39-bb82-618221db4fb9 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:12.623004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5806b4-7854-4197-b088-a1ae8eda3e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:12.624237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5806b4-7854-4197-b088-a1ae8eda3e23 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:12.624743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5806b4-7854-4197-b088-a1ae8eda3e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:12.625237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5806b4-7854-4197-b088-a1ae8eda3e23 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:12.626598 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5806b4-7854-4197-b088-a1ae8eda3e23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:14:12.628146 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5806b4-7854-4197-b088-a1ae8eda3e23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:12.632012 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5806b4-7854-4197-b088-a1ae8eda3e23 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:12.651782 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1076/2164] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:14:12.657693 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52828178-7143-4488-9ceb-478e8be840bd None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:12.677275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52828178-7143-4488-9ceb-478e8be840bd None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:12.688819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a180e3-6541-4f4f-98b0-0816cc382e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:12.690826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a180e3-6541-4f4f-98b0-0816cc382e20 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:12.691231 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a180e3-6541-4f4f-98b0-0816cc382e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:12.691624 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a180e3-6541-4f4f-98b0-0816cc382e20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:12.692889 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a180e3-6541-4f4f-98b0-0816cc382e20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:12.696710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a180e3-6541-4f4f-98b0-0816cc382e20 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:12.742755 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1089/2165] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:14:12 2022] POST /identity/v3/projects => generated 371 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:14:12.749258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5806b4-7854-4197-b088-a1ae8eda3e23 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:12.762112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5806b4-7854-4197-b088-a1ae8eda3e23 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:12.868141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1b025-1470-4641-be67-efde70e414f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:12.869486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b025-1470-4641-be67-efde70e414f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:12.870373 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b025-1470-4641-be67-efde70e414f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:12.870845 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b025-1470-4641-be67-efde70e414f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:12.871728 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1b025-1470-4641-be67-efde70e414f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:12.874343 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1b025-1470-4641-be67-efde70e414f8 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:12.905297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1077/2166] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:14:12 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:14:12.911132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a180e3-6541-4f4f-98b0-0816cc382e20 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:12.926471 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a180e3-6541-4f4f-98b0-0816cc382e20 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:12.932713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ba4ef-1ab6-4989-a19f-ae34b8eb9ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:12.933552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ba4ef-1ab6-4989-a19f-ae34b8eb9ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:12.934125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ba4ef-1ab6-4989-a19f-ae34b8eb9ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:12.934461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ba4ef-1ab6-4989-a19f-ae34b8eb9ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:12.935541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ba4ef-1ab6-4989-a19f-ae34b8eb9ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:12.937523 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ba4ef-1ab6-4989-a19f-ae34b8eb9ee1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:12.952654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1090/2167] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:14:12.957454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1b025-1470-4641-be67-efde70e414f8 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:12.969616 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1b025-1470-4641-be67-efde70e414f8 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:12.976314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a412a22-e7c3-4244-8a00-75738f14adf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:12.977142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a412a22-e7c3-4244-8a00-75738f14adf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:12.977507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a412a22-e7c3-4244-8a00-75738f14adf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:12.977878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a412a22-e7c3-4244-8a00-75738f14adf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:12.978950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a412a22-e7c3-4244-8a00-75738f14adf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:12.980872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a412a22-e7c3-4244-8a00-75738f14adf8 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:12.995097 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1078/2168] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:14:13.000125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ba4ef-1ab6-4989-a19f-ae34b8eb9ee1 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:13.019289 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ba4ef-1ab6-4989-a19f-ae34b8eb9ee1 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:13.027235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611144ed-0272-4cdb-8c5a-6d93a7c01eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:13.028504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611144ed-0272-4cdb-8c5a-6d93a7c01eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:13.028802 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611144ed-0272-4cdb-8c5a-6d93a7c01eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:13.029218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611144ed-0272-4cdb-8c5a-6d93a7c01eeb admin admin] PATH_INFO: `/v3/projects/6a2b0fce04704afa812423f6fee21c84/users/5181b551e8e541c88dcee59ea47d20eb/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:13.059792 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611144ed-0272-4cdb-8c5a-6d93a7c01eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a2b0fce04704afa812423f6fee21c84, user_id=5181b551e8e541c88dcee59ea47d20eb, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:13.062676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611144ed-0272-4cdb-8c5a-6d93a7c01eeb admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:13.079879 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1091/2169] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:14:12 2022] PUT /identity/v3/projects/6a2b0fce04704afa812423f6fee21c84/users/5181b551e8e541c88dcee59ea47d20eb/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:14:13.086563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82524e37-b6dc-4dc5-ad82-7ffd9e7faee0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:13.086915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82524e37-b6dc-4dc5-ad82-7ffd9e7faee0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:13.087210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82524e37-b6dc-4dc5-ad82-7ffd9e7faee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:13.110400 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-82524e37-b6dc-4dc5-ad82-7ffd9e7faee0 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:14:13.113241 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-82524e37-b6dc-4dc5-ad82-7ffd9e7faee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:13.188328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:14:13.209203 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1079/2170] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:14:13.221560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611144ed-0272-4cdb-8c5a-6d93a7c01eeb admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:13.223822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-611144ed-0272-4cdb-8c5a-6d93a7c01eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:13.324394 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611144ed-0272-4cdb-8c5a-6d93a7c01eeb admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:13.328705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5db51-3f8c-490c-a59c-73428c2c7989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:13.329519 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5db51-3f8c-490c-a59c-73428c2c7989 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:13.329870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5db51-3f8c-490c-a59c-73428c2c7989 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:13.330226 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5db51-3f8c-490c-a59c-73428c2c7989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:13.343635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5db51-3f8c-490c-a59c-73428c2c7989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:13.346518 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5db51-3f8c-490c-a59c-73428c2c7989 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:13.368565 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1092/2171] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:14:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:14:14.293469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b10c27-1a79-409d-b506-d8f9b23aba27 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:14.294305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b10c27-1a79-409d-b506-d8f9b23aba27 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:14.294305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b10c27-1a79-409d-b506-d8f9b23aba27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:14.326730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-87b10c27-1a79-409d-b506-d8f9b23aba27 None None] MFA Rules not processed for user `5181b551e8e541c88dcee59ea47d20eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:14:14.328017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-87b10c27-1a79-409d-b506-d8f9b23aba27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:14.426508 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1080/2172] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:14:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:14:14.438349 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5db51-3f8c-490c-a59c-73428c2c7989 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:14:14.440037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-32e5db51-3f8c-490c-a59c-73428c2c7989 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:14.445927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:14:14.511066 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5db51-3f8c-490c-a59c-73428c2c7989 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:14:14.514978 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f9a56-5c46-4625-8fa4-439d83382a2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:14:14.515658 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f9a56-5c46-4625-8fa4-439d83382a2e service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:14.515975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f9a56-5c46-4625-8fa4-439d83382a2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:14.516263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f9a56-5c46-4625-8fa4-439d83382a2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:14.526154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f9a56-5c46-4625-8fa4-439d83382a2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:14:14.527637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f9a56-5c46-4625-8fa4-439d83382a2e service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:14:14.539369 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1093/2173] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:14:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:14:17.115369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750e2cb-d59d-45cc-bf8a-dc91faea0ac0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:17.116153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750e2cb-d59d-45cc-bf8a-dc91faea0ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:17.116153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750e2cb-d59d-45cc-bf8a-dc91faea0ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:17.146272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2750e2cb-d59d-45cc-bf8a-dc91faea0ac0 None None] MFA Rules not processed for user `5181b551e8e541c88dcee59ea47d20eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:14:17.148552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2750e2cb-d59d-45cc-bf8a-dc91faea0ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:17.161592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1081/2174] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:14:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:14:24.534105 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=25 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:14:27.137376 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=36 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:14:42.622053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa858c-eeff-4aab-82e6-e0bfb8cc072f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:42.622663 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa858c-eeff-4aab-82e6-e0bfb8cc072f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:42.622787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa858c-eeff-4aab-82e6-e0bfb8cc072f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:42.644844 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b4aa858c-eeff-4aab-82e6-e0bfb8cc072f None None] MFA Rules not processed for user `326bf42fe8c642b4898f2c8380733599`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:14:42.646146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b4aa858c-eeff-4aab-82e6-e0bfb8cc072f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:42.715073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1094/2175] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:14:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:14:48.780795 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea98ec9-9d7d-4684-92df-262eb2e3ecfd None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:48.782014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea98ec9-9d7d-4684-92df-262eb2e3ecfd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:48.782014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea98ec9-9d7d-4684-92df-262eb2e3ecfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:48.807763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4ea98ec9-9d7d-4684-92df-262eb2e3ecfd None None] MFA Rules not processed for user `5181b551e8e541c88dcee59ea47d20eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:14:48.810213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4ea98ec9-9d7d-4684-92df-262eb2e3ecfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:48.823139 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1082/2176] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:14:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:14:52.705410 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=25 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:14:57.817453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51161d92-f559-4e1a-9da8-5014bd3087fa None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:14:57.818129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51161d92-f559-4e1a-9da8-5014bd3087fa None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:14:57.818129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51161d92-f559-4e1a-9da8-5014bd3087fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:14:57.838937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-51161d92-f559-4e1a-9da8-5014bd3087fa None None] MFA Rules not processed for user `7b4006c43f7944c083cf323db4df2b3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:14:57.840390 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-51161d92-f559-4e1a-9da8-5014bd3087fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:14:57.931513 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1095/2177] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:14:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:14:58.796416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:15:02.396692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea98ec9-9d7d-4684-92df-262eb2e3ecfd None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:15:02.398151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4ea98ec9-9d7d-4684-92df-262eb2e3ecfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:15:02.482174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea98ec9-9d7d-4684-92df-262eb2e3ecfd None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:15:02.489183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c474b4b-1817-4eb1-98e7-922774774db4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:15:02.489857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c474b4b-1817-4eb1-98e7-922774774db4 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:15:02.490142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c474b4b-1817-4eb1-98e7-922774774db4 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:15:02.490425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c474b4b-1817-4eb1-98e7-922774774db4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:15:02.492386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5c474b4b-1817-4eb1-98e7-922774774db4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:15:02.504562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c474b4b-1817-4eb1-98e7-922774774db4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:15:02.506238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c474b4b-1817-4eb1-98e7-922774774db4 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:15:02.518136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1083/2178] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:15:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:15:07.921397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=25 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:15:12.512230 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=34 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:15:13.183130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51161d92-f559-4e1a-9da8-5014bd3087fa None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:15:13.185592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-51161d92-f559-4e1a-9da8-5014bd3087fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:15:13.260493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51161d92-f559-4e1a-9da8-5014bd3087fa None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:15:13.264441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9fce4-b7dc-4a85-9365-cd9390247be3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e053b6b25a4e4a20a05e6ffa6683222b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:15:13.265634 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9fce4-b7dc-4a85-9365-cd9390247be3 service glance] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:15:13.265918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9fce4-b7dc-4a85-9365-cd9390247be3 service glance] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:15:13.266320 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9fce4-b7dc-4a85-9365-cd9390247be3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:15:13.268496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4ed9fce4-b7dc-4a85-9365-cd9390247be3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:15:13.284410 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9fce4-b7dc-4a85-9365-cd9390247be3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:15:13.285862 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9fce4-b7dc-4a85-9365-cd9390247be3 service glance] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:15:13.298628 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1096/2179] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:15:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:15:13.518172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c474b4b-1817-4eb1-98e7-922774774db4 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:15:13.536183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c474b4b-1817-4eb1-98e7-922774774db4 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:15:13.551438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733bbcb-a65c-48b1-b702-0fee8c0eb285 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:15:13.552540 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733bbcb-a65c-48b1-b702-0fee8c0eb285 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:15:13.553288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733bbcb-a65c-48b1-b702-0fee8c0eb285 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:15:13.554337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733bbcb-a65c-48b1-b702-0fee8c0eb285 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:15:13.559506 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5733bbcb-a65c-48b1-b702-0fee8c0eb285 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:15:13.596954 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733bbcb-a65c-48b1-b702-0fee8c0eb285 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:15:13.599673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733bbcb-a65c-48b1-b702-0fee8c0eb285 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:15:13.623754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1084/2180] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:15:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:15:17.647309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9fce4-b7dc-4a85-9365-cd9390247be3 service glance] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:15:17.667605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9fce4-b7dc-4a85-9365-cd9390247be3 service glance] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:15:17.678189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd0ec6-f161-4dc6-a674-1604d37cca5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:15:17.679348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd0ec6-f161-4dc6-a674-1604d37cca5e service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:15:17.679864 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd0ec6-f161-4dc6-a674-1604d37cca5e service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:15:17.680424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd0ec6-f161-4dc6-a674-1604d37cca5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:15:17.683756 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-09dd0ec6-f161-4dc6-a674-1604d37cca5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:15:17.782314 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd0ec6-f161-4dc6-a674-1604d37cca5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:15:17.784053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd0ec6-f161-4dc6-a674-1604d37cca5e service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:15:17.800052 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1097/2181] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:15:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:15:23.610296 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=6 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:15:27.789755 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=54 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:15:41.136349 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca39037-3883-4f1f-b943-6d415e3ab369 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:15:41.136870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca39037-3883-4f1f-b943-6d415e3ab369 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:15:41.136969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca39037-3883-4f1f-b943-6d415e3ab369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:15:41.156191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-9ca39037-3883-4f1f-b943-6d415e3ab369 None None] MFA Rules not processed for user `257450db238d4efda783e30cbcd23d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:15:41.157416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9ca39037-3883-4f1f-b943-6d415e3ab369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:15:41.216356 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1085/2182] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jul 21 09:15:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:15:41.259272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd0ec6-f161-4dc6-a674-1604d37cca5e service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:15:41.261305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-09dd0ec6-f161-4dc6-a674-1604d37cca5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:15:41.343495 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd0ec6-f161-4dc6-a674-1604d37cca5e service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:15:41.348972 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df41d05-87d4-421c-9638-c55050246957 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061def74ec7f42ce9a174e0944bf499f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:15:41.349690 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df41d05-87d4-421c-9638-c55050246957 service swift] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:15:41.349974 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df41d05-87d4-421c-9638-c55050246957 service swift] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:15:41.350250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df41d05-87d4-421c-9638-c55050246957 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:15:41.360627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df41d05-87d4-421c-9638-c55050246957 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:15:41.362292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df41d05-87d4-421c-9638-c55050246957 service swift] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:15:41.371440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1098/2183] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jul 21 09:15:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 09:15:46.165585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca39037-3883-4f1f-b943-6d415e3ab369 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:15:46.180619 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca39037-3883-4f1f-b943-6d415e3ab369 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:15:46.186753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23684033-15ff-4afa-b8cd-34d67a4c2473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:15:46.188013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23684033-15ff-4afa-b8cd-34d67a4c2473 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:15:46.189164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23684033-15ff-4afa-b8cd-34d67a4c2473 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:15:46.189676 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23684033-15ff-4afa-b8cd-34d67a4c2473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:15:46.193058 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-23684033-15ff-4afa-b8cd-34d67a4c2473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:15:46.213714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23684033-15ff-4afa-b8cd-34d67a4c2473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:15:46.216869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23684033-15ff-4afa-b8cd-34d67a4c2473 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:15:46.235248 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1086/2184] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:15:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:15:51.367870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=32 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:15:56.226629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=31 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:15:57.147640 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df41d05-87d4-421c-9638-c55050246957 service swift] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:15:57.149156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7df41d05-87d4-421c-9638-c55050246957 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:15:57.166645 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df41d05-87d4-421c-9638-c55050246957 service swift] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:15:57.174617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecdf4d-7ef9-4602-a747-51fd3afb0fdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:15:57.175732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecdf4d-7ef9-4602-a747-51fd3afb0fdf service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:15:57.176387 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecdf4d-7ef9-4602-a747-51fd3afb0fdf service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:15:57.176867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecdf4d-7ef9-4602-a747-51fd3afb0fdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:15:57.180155 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-88ecdf4d-7ef9-4602-a747-51fd3afb0fdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:15:57.196482 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ecdf4d-7ef9-4602-a747-51fd3afb0fdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:15:57.198176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ecdf4d-7ef9-4602-a747-51fd3afb0fdf service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:15:57.211119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1099/2185] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:15:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:16:07.205571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:16:16.000662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23684033-15ff-4afa-b8cd-34d67a4c2473 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:16:16.015207 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23684033-15ff-4afa-b8cd-34d67a4c2473 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:16:16.020738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e594c5-b568-4155-8204-31acc0bd0c35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:16:16.021384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e594c5-b568-4155-8204-31acc0bd0c35 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:16:16.021672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e594c5-b568-4155-8204-31acc0bd0c35 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:16:16.021952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e594c5-b568-4155-8204-31acc0bd0c35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:16:16.023771 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-80e594c5-b568-4155-8204-31acc0bd0c35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:16:16.093632 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e594c5-b568-4155-8204-31acc0bd0c35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:16:16.095304 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e594c5-b568-4155-8204-31acc0bd0c35 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:16:16.105472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1087/2186] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:16:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:16:26.100947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=28 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:17:15.772912 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecdf4d-7ef9-4602-a747-51fd3afb0fdf service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:17:15.774381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-88ecdf4d-7ef9-4602-a747-51fd3afb0fdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:17:15.786134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecdf4d-7ef9-4602-a747-51fd3afb0fdf service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:17:15.790567 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850988b-4b55-4ef6-939d-c7a0c86e7658 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:17:15.791214 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850988b-4b55-4ef6-939d-c7a0c86e7658 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:17:15.791500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850988b-4b55-4ef6-939d-c7a0c86e7658 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:17:15.791783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850988b-4b55-4ef6-939d-c7a0c86e7658 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:17:15.793654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b850988b-4b55-4ef6-939d-c7a0c86e7658 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:17:15.856402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850988b-4b55-4ef6-939d-c7a0c86e7658 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:17:15.862001 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850988b-4b55-4ef6-939d-c7a0c86e7658 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:17:15.869695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1100/2187] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:17:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:17:16.501115 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1d440-601f-43fc-9af6-a6a851fdb887 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:17:16.501654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1d440-601f-43fc-9af6-a6a851fdb887 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:17:16.502075 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1d440-601f-43fc-9af6-a6a851fdb887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:17:16.524701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-68e1d440-601f-43fc-9af6-a6a851fdb887 None None] MFA Rules not processed for user `257450db238d4efda783e30cbcd23d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:17:16.526178 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-68e1d440-601f-43fc-9af6-a6a851fdb887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:17:16.536232 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1088/2188] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jul 21 09:17:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:17:16.609723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850988b-4b55-4ef6-939d-c7a0c86e7658 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:17:16.627534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850988b-4b55-4ef6-939d-c7a0c86e7658 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:17:16.641495 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb816a1c-cc55-467e-994e-0f226bb742e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061def74ec7f42ce9a174e0944bf499f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:17:16.642432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb816a1c-cc55-467e-994e-0f226bb742e1 service swift] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:17:16.643276 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb816a1c-cc55-467e-994e-0f226bb742e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:17:16.643571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb816a1c-cc55-467e-994e-0f226bb742e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:17:16.655274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb816a1c-cc55-467e-994e-0f226bb742e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:17:16.656873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb816a1c-cc55-467e-994e-0f226bb742e1 service swift] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:17:16.676180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1101/2189] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jul 21 09:17:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 09:17:18.199767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1d440-601f-43fc-9af6-a6a851fdb887 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:17:18.214763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1d440-601f-43fc-9af6-a6a851fdb887 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:17:18.220753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb065743-1bc0-47e1-b5b5-dafcca8f4539 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:17:18.221531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb065743-1bc0-47e1-b5b5-dafcca8f4539 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:17:18.221880 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb065743-1bc0-47e1-b5b5-dafcca8f4539 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:17:18.222168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb065743-1bc0-47e1-b5b5-dafcca8f4539 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:17:18.225426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bb065743-1bc0-47e1-b5b5-dafcca8f4539 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:17:18.239889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb065743-1bc0-47e1-b5b5-dafcca8f4539 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:17:18.242509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb065743-1bc0-47e1-b5b5-dafcca8f4539 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:17:18.255365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1089/2190] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:17:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:17:20.093242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d1693-5433-4a7e-875a-e3547b6b860c None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:17:20.093765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d1693-5433-4a7e-875a-e3547b6b860c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:17:20.093765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d1693-5433-4a7e-875a-e3547b6b860c None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:17:20.095061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1102/2191] 10.222.0.13 () {54 vars in 912 bytes} [Thu Jul 21 09:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:17:20.101509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ee822-6f61-4b99-b265-f496eb77c82a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:17:20.101989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ee822-6f61-4b99-b265-f496eb77c82a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:17:20.102203 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ee822-6f61-4b99-b265-f496eb77c82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:17:20.122111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-370ee822-6f61-4b99-b265-f496eb77c82a None None] MFA Rules not processed for user `72f9ef6de66e456e800237e9f954e98d`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:17:20.123338 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-370ee822-6f61-4b99-b265-f496eb77c82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:17:20.189448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1090/2192] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jul 21 09:17:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:17:20.214106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d1693-5433-4a7e-875a-e3547b6b860c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:17:20.216183 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-260d1693-5433-4a7e-875a-e3547b6b860c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:17:20.233886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d1693-5433-4a7e-875a-e3547b6b860c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:17:20.239788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461880dc-71ae-4110-9085-742617864416 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:17:20.240582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461880dc-71ae-4110-9085-742617864416 service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:17:20.240860 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461880dc-71ae-4110-9085-742617864416 service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:17:20.241135 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461880dc-71ae-4110-9085-742617864416 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:17:20.250507 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461880dc-71ae-4110-9085-742617864416 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:17:20.252140 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461880dc-71ae-4110-9085-742617864416 service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:17:20.262706 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1103/2193] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:17:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:17:22.155368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ee822-6f61-4b99-b265-f496eb77c82a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:17:22.169324 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ee822-6f61-4b99-b265-f496eb77c82a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:17:22.176394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16f7bb-3b1e-4956-81b2-892008d8deb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:17:22.177759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16f7bb-3b1e-4956-81b2-892008d8deb6 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:17:22.178270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16f7bb-3b1e-4956-81b2-892008d8deb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:17:22.178620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16f7bb-3b1e-4956-81b2-892008d8deb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:17:22.180631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1f16f7bb-3b1e-4956-81b2-892008d8deb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:17:22.196043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16f7bb-3b1e-4956-81b2-892008d8deb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:17:22.198041 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16f7bb-3b1e-4956-81b2-892008d8deb6 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:17:22.217998 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1091/2194] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:17:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:17:23.149612 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2921ff-8689-46a1-a6bd-5e22259ac316 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:17:23.150196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2921ff-8689-46a1-a6bd-5e22259ac316 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:17:23.150196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2921ff-8689-46a1-a6bd-5e22259ac316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:17:23.170124 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ef2921ff-8689-46a1-a6bd-5e22259ac316 None None] MFA Rules not processed for user `e717fb256de24adb8ee6815338a94e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:17:23.171626 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ef2921ff-8689-46a1-a6bd-5e22259ac316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:17:23.179874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1104/2195] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:17:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:17:23.526804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16f7bb-3b1e-4956-81b2-892008d8deb6 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:17:23.528584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1f16f7bb-3b1e-4956-81b2-892008d8deb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:17:23.541928 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16f7bb-3b1e-4956-81b2-892008d8deb6 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:17:23.545836 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835f030-97ba-450e-ac4f-a3269a2d515c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:17:23.546502 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835f030-97ba-450e-ac4f-a3269a2d515c service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:17:23.546797 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835f030-97ba-450e-ac4f-a3269a2d515c service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:17:23.547070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835f030-97ba-450e-ac4f-a3269a2d515c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:17:23.549242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1835f030-97ba-450e-ac4f-a3269a2d515c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:17:23.560697 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835f030-97ba-450e-ac4f-a3269a2d515c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:17:23.562169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835f030-97ba-450e-ac4f-a3269a2d515c service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:17:23.572072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1092/2196] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:17:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:17:23.900663 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2921ff-8689-46a1-a6bd-5e22259ac316 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:17:23.902475 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ef2921ff-8689-46a1-a6bd-5e22259ac316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:17:23.921204 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2921ff-8689-46a1-a6bd-5e22259ac316 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:17:23.926452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca301f-3c60-46ea-8935-ad824cc63336 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:17:23.927146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca301f-3c60-46ea-8935-ad824cc63336 service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:17:23.927436 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca301f-3c60-46ea-8935-ad824cc63336 service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:17:23.927739 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca301f-3c60-46ea-8935-ad824cc63336 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:17:23.929573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-dfca301f-3c60-46ea-8935-ad824cc63336 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:17:23.944585 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca301f-3c60-46ea-8935-ad824cc63336 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:17:23.946562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca301f-3c60-46ea-8935-ad824cc63336 service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:17:23.962186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1105/2197] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:17:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:17:33.568407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=50 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:17:33.958390 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=49 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:17:35.195641 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835f030-97ba-450e-ac4f-a3269a2d515c service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:17:35.208878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835f030-97ba-450e-ac4f-a3269a2d515c service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:17:35.214211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e08ae-1ce6-40bc-96a0-556ebd72f690 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:17:35.214869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e08ae-1ce6-40bc-96a0-556ebd72f690 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:17:35.215148 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e08ae-1ce6-40bc-96a0-556ebd72f690 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:17:35.215418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e08ae-1ce6-40bc-96a0-556ebd72f690 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:17:35.217214 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-425e08ae-1ce6-40bc-96a0-556ebd72f690 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:17:35.276047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e08ae-1ce6-40bc-96a0-556ebd72f690 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:17:35.277664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e08ae-1ce6-40bc-96a0-556ebd72f690 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:17:35.287541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1093/2198] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:17:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:17:45.282454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=28 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:18:40.825432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca301f-3c60-46ea-8935-ad824cc63336 service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:18:40.890132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca301f-3c60-46ea-8935-ad824cc63336 service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:18:40.896023 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f36f7-7b8d-4e3d-bdc7-59b536289636 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:18:40.896705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f36f7-7b8d-4e3d-bdc7-59b536289636 service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:18:40.896991 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f36f7-7b8d-4e3d-bdc7-59b536289636 service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:18:40.897268 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f36f7-7b8d-4e3d-bdc7-59b536289636 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:18:40.899343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fb4f36f7-7b8d-4e3d-bdc7-59b536289636 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:18:40.937006 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f36f7-7b8d-4e3d-bdc7-59b536289636 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:18:40.938617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f36f7-7b8d-4e3d-bdc7-59b536289636 service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:18:40.950185 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1106/2199] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:18:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:18:50.945805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=36 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:19:16.688312 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e08ae-1ce6-40bc-96a0-556ebd72f690 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:19:16.724738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e08ae-1ce6-40bc-96a0-556ebd72f690 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:19:16.728426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fdd60-b7cf-4982-baea-1a5957830c94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:19:16.729064 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fdd60-b7cf-4982-baea-1a5957830c94 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:19:16.729344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fdd60-b7cf-4982-baea-1a5957830c94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:19:16.729620 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fdd60-b7cf-4982-baea-1a5957830c94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:19:16.738738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fdd60-b7cf-4982-baea-1a5957830c94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:19:16.740293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fdd60-b7cf-4982-baea-1a5957830c94 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:19:16.749611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1094/2200] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:19:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:19:16.844538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f36f7-7b8d-4e3d-bdc7-59b536289636 service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:19:16.855645 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f36f7-7b8d-4e3d-bdc7-59b536289636 service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:19:16.859343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d0930-b2a4-4e56-b7fc-e1243412d1dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:19:16.860059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d0930-b2a4-4e56-b7fc-e1243412d1dc service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:19:16.860341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d0930-b2a4-4e56-b7fc-e1243412d1dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:19:16.860611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d0930-b2a4-4e56-b7fc-e1243412d1dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:19:16.862478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b25d0930-b2a4-4e56-b7fc-e1243412d1dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:19:16.874773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d0930-b2a4-4e56-b7fc-e1243412d1dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:19:16.876529 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d0930-b2a4-4e56-b7fc-e1243412d1dc service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:19:16.887064 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1107/2201] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:19:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:19:26.745445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=15 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:19:26.883132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:20:03.002247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fdd60-b7cf-4982-baea-1a5957830c94 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:20:03.033824 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fdd60-b7cf-4982-baea-1a5957830c94 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:20:03.039910 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bcea9-9b4a-4d72-866b-683f45de1437 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:20:03.040636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bcea9-9b4a-4d72-866b-683f45de1437 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:20:03.041317 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bcea9-9b4a-4d72-866b-683f45de1437 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:20:03.041317 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bcea9-9b4a-4d72-866b-683f45de1437 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:20:03.050079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bcea9-9b4a-4d72-866b-683f45de1437 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:20:03.051682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bcea9-9b4a-4d72-866b-683f45de1437 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:20:03.081941 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1095/2202] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:20:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:20:13.079273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=20 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:20:14.461275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d0930-b2a4-4e56-b7fc-e1243412d1dc service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:20:14.482100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d0930-b2a4-4e56-b7fc-e1243412d1dc service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:20:14.485729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ae484-5f04-4d39-ac6d-a552684fe712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:20:14.486367 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ae484-5f04-4d39-ac6d-a552684fe712 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:20:14.486647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ae484-5f04-4d39-ac6d-a552684fe712 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:20:14.486923 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ae484-5f04-4d39-ac6d-a552684fe712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:20:14.498189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ae484-5f04-4d39-ac6d-a552684fe712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:20:14.499637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ae484-5f04-4d39-ac6d-a552684fe712 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:20:14.511633 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1108/2203] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:20:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:20:24.506896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=10 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:20:54.087567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bcea9-9b4a-4d72-866b-683f45de1437 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:20:54.101191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bcea9-9b4a-4d72-866b-683f45de1437 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:20:54.106340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5340a6-f35d-42c8-b765-3cf4858c129b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:20:54.107235 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5340a6-f35d-42c8-b765-3cf4858c129b service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:20:54.107746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5340a6-f35d-42c8-b765-3cf4858c129b service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:20:54.108284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5340a6-f35d-42c8-b765-3cf4858c129b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:20:54.121619 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5340a6-f35d-42c8-b765-3cf4858c129b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:20:54.124329 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5340a6-f35d-42c8-b765-3cf4858c129b service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:20:54.140307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1096/2204] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:20:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:21:04.132543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=6 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:21:18.902346 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ae484-5f04-4d39-ac6d-a552684fe712 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:21:18.911514 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ae484-5f04-4d39-ac6d-a552684fe712 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:21:18.915090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ba6b9-cb51-4afe-bebc-d07f7cead8da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:21:18.915727 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ba6b9-cb51-4afe-bebc-d07f7cead8da service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:21:18.916051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ba6b9-cb51-4afe-bebc-d07f7cead8da service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:21:18.916328 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ba6b9-cb51-4afe-bebc-d07f7cead8da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:21:18.926106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ba6b9-cb51-4afe-bebc-d07f7cead8da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:21:18.927635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ba6b9-cb51-4afe-bebc-d07f7cead8da service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:21:18.956946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1109/2205] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:21:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:21:19.002977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5340a6-f35d-42c8-b765-3cf4858c129b service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:21:19.013243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5340a6-f35d-42c8-b765-3cf4858c129b service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:21:19.016872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f743e6-9175-4e4f-bb88-0e45d9d9cb3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:21:19.017509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f743e6-9175-4e4f-bb88-0e45d9d9cb3d service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:21:19.017793 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f743e6-9175-4e4f-bb88-0e45d9d9cb3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:21:19.018070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f743e6-9175-4e4f-bb88-0e45d9d9cb3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:21:19.027136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f743e6-9175-4e4f-bb88-0e45d9d9cb3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:21:19.028722 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f743e6-9175-4e4f-bb88-0e45d9d9cb3d service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:21:19.057155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1097/2206] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:21:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:21:19.910188 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ba6b9-cb51-4afe-bebc-d07f7cead8da service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:21:19.919398 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ba6b9-cb51-4afe-bebc-d07f7cead8da service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:21:19.923158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8993007-4ea5-42d9-9b23-c848621a70f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:21:19.923784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8993007-4ea5-42d9-9b23-c848621a70f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:21:19.924104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8993007-4ea5-42d9-9b23-c848621a70f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:21:19.924381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8993007-4ea5-42d9-9b23-c848621a70f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:21:19.934657 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8993007-4ea5-42d9-9b23-c848621a70f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:21:19.936114 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8993007-4ea5-42d9-9b23-c848621a70f7 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:21:19.946745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1110/2207] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:21:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:21:27.049145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b1244-ad2e-4201-868f-f7949c7e387a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:21:27.049699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b1244-ad2e-4201-868f-f7949c7e387a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:21:27.049699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b1244-ad2e-4201-868f-f7949c7e387a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:21:27.098957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e68b1244-ad2e-4201-868f-f7949c7e387a None None] MFA Rules not processed for user `6c62db4b38434cfe886108a72e4c42aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:21:27.100216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e68b1244-ad2e-4201-868f-f7949c7e387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:21:27.120623 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1098/2208] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:21:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:21:27.437424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8993007-4ea5-42d9-9b23-c848621a70f7 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:21:27.445823 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8993007-4ea5-42d9-9b23-c848621a70f7 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:21:27.450187 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292cd2f-6615-4e7f-981c-783878fa356a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:21:27.455971 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292cd2f-6615-4e7f-981c-783878fa356a service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:21:27.455971 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292cd2f-6615-4e7f-981c-783878fa356a service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:21:27.455971 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292cd2f-6615-4e7f-981c-783878fa356a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:21:27.455971 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2292cd2f-6615-4e7f-981c-783878fa356a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:21:27.477661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292cd2f-6615-4e7f-981c-783878fa356a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:21:27.479106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292cd2f-6615-4e7f-981c-783878fa356a service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:21:27.487968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1111/2209] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:21:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:21:36.893158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b1244-ad2e-4201-868f-f7949c7e387a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:21:36.915822 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b1244-ad2e-4201-868f-f7949c7e387a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:21:36.924806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641d34a-6733-41a0-beaf-0b1192af7438 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:21:36.925772 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641d34a-6733-41a0-beaf-0b1192af7438 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:21:36.926198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641d34a-6733-41a0-beaf-0b1192af7438 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:21:36.926633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641d34a-6733-41a0-beaf-0b1192af7438 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:21:36.938974 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641d34a-6733-41a0-beaf-0b1192af7438 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:21:36.941155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641d34a-6733-41a0-beaf-0b1192af7438 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:21:36.952631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1099/2210] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:21:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:21:37.484084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=30 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:21:46.946741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=38 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:22:34.940299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292cd2f-6615-4e7f-981c-783878fa356a service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:22:34.953459 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292cd2f-6615-4e7f-981c-783878fa356a service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:22:34.959682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9798caa-c357-4db2-95e2-e4066d164e34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:22:34.960366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9798caa-c357-4db2-95e2-e4066d164e34 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:22:34.960654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9798caa-c357-4db2-95e2-e4066d164e34 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:22:34.960937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9798caa-c357-4db2-95e2-e4066d164e34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:22:34.969166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9798caa-c357-4db2-95e2-e4066d164e34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:22:34.970697 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9798caa-c357-4db2-95e2-e4066d164e34 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:22:34.998908 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1112/2211] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:22:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:22:44.995953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=13 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:22:55.959319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641d34a-6733-41a0-beaf-0b1192af7438 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:22:55.970850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641d34a-6733-41a0-beaf-0b1192af7438 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:22:55.976921 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d92b49-2a55-4952-b92d-4570eaf7df81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:22:55.977588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d92b49-2a55-4952-b92d-4570eaf7df81 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:22:55.977872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d92b49-2a55-4952-b92d-4570eaf7df81 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:22:55.978800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d92b49-2a55-4952-b92d-4570eaf7df81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:22:55.987845 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d92b49-2a55-4952-b92d-4570eaf7df81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:22:55.989515 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d92b49-2a55-4952-b92d-4570eaf7df81 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:22:55.999724 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1100/2212] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:22:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:23:05.995090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=6 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:23:21.189013 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9798caa-c357-4db2-95e2-e4066d164e34 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:23:21.200204 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9798caa-c357-4db2-95e2-e4066d164e34 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:23:21.206002 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88556bdd-dd76-4754-9dff-25da66a3378c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:23:21.207144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88556bdd-dd76-4754-9dff-25da66a3378c service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:23:21.207423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88556bdd-dd76-4754-9dff-25da66a3378c service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:23:21.207696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88556bdd-dd76-4754-9dff-25da66a3378c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:23:21.217291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88556bdd-dd76-4754-9dff-25da66a3378c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:23:21.218913 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88556bdd-dd76-4754-9dff-25da66a3378c service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:23:21.229780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1113/2213] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:23:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:23:31.225199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:23:49.415450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d92b49-2a55-4952-b92d-4570eaf7df81 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:23:49.427583 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d92b49-2a55-4952-b92d-4570eaf7df81 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:23:49.432718 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03959a-760a-4d69-aee9-52bc30d857fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:23:49.433362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03959a-760a-4d69-aee9-52bc30d857fd service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:23:49.433644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03959a-760a-4d69-aee9-52bc30d857fd service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:23:49.433947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03959a-760a-4d69-aee9-52bc30d857fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:23:49.442807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03959a-760a-4d69-aee9-52bc30d857fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:23:49.444412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03959a-760a-4d69-aee9-52bc30d857fd service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:23:49.453986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1101/2214] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:23:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:23:59.449935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=6 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:25:02.690617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88556bdd-dd76-4754-9dff-25da66a3378c service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:25:02.767936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88556bdd-dd76-4754-9dff-25da66a3378c service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:25:02.774129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45b0c8-7481-49c1-a55c-98be68de19ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:25:02.774798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45b0c8-7481-49c1-a55c-98be68de19ff service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:25:02.775077 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45b0c8-7481-49c1-a55c-98be68de19ff service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:25:02.775356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45b0c8-7481-49c1-a55c-98be68de19ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:25:02.777311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0e45b0c8-7481-49c1-a55c-98be68de19ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:25:02.850149 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45b0c8-7481-49c1-a55c-98be68de19ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:25:02.851763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45b0c8-7481-49c1-a55c-98be68de19ff service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:25:02.861490 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1114/2215] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:25:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:25:12.856906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=61 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:25:19.153914 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03959a-760a-4d69-aee9-52bc30d857fd service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:25:19.210685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03959a-760a-4d69-aee9-52bc30d857fd service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:25:19.214359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e4e3a-4eba-4c85-b9da-70b5fd026229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:25:19.214992 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e4e3a-4eba-4c85-b9da-70b5fd026229 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:25:19.215278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e4e3a-4eba-4c85-b9da-70b5fd026229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:25:19.215555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e4e3a-4eba-4c85-b9da-70b5fd026229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:25:19.217496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-909e4e3a-4eba-4c85-b9da-70b5fd026229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:25:19.243857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e4e3a-4eba-4c85-b9da-70b5fd026229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:25:19.245335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e4e3a-4eba-4c85-b9da-70b5fd026229 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:25:19.257019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1102/2216] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:25:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:25:19.912351 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45b0c8-7481-49c1-a55c-98be68de19ff service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:25:19.969640 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45b0c8-7481-49c1-a55c-98be68de19ff service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:25:19.974143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487b234-e052-442c-b8b5-967aefe6cdc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:25:19.974789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b234-e052-442c-b8b5-967aefe6cdc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:25:19.975123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b234-e052-442c-b8b5-967aefe6cdc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:25:19.975402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b234-e052-442c-b8b5-967aefe6cdc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:25:19.977326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e487b234-e052-442c-b8b5-967aefe6cdc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:25:20.056555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e487b234-e052-442c-b8b5-967aefe6cdc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:25:20.058006 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e487b234-e052-442c-b8b5-967aefe6cdc2 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:25:20.076014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1115/2217] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:25:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:25:20.160097 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e4e3a-4eba-4c85-b9da-70b5fd026229 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:25:20.170004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e4e3a-4eba-4c85-b9da-70b5fd026229 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:25:20.174114 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dba653-a893-4a2b-b9f9-21f8fb822507 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:25:20.174759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dba653-a893-4a2b-b9f9-21f8fb822507 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:25:20.175046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dba653-a893-4a2b-b9f9-21f8fb822507 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:25:20.175322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dba653-a893-4a2b-b9f9-21f8fb822507 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:25:20.241833 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dba653-a893-4a2b-b9f9-21f8fb822507 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:25:20.243219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dba653-a893-4a2b-b9f9-21f8fb822507 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:25:20.264181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1103/2218] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:25:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:25:30.073525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=61 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:25:30.259482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=67 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:26:01.096060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487b234-e052-442c-b8b5-967aefe6cdc2 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:26:01.098548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e487b234-e052-442c-b8b5-967aefe6cdc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:26:01.112948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487b234-e052-442c-b8b5-967aefe6cdc2 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:26:01.116940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6987c8-7270-4948-a758-0f8e43142e9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:26:01.117594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6987c8-7270-4948-a758-0f8e43142e9a service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:26:01.117881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6987c8-7270-4948-a758-0f8e43142e9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:26:01.118159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6987c8-7270-4948-a758-0f8e43142e9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:26:01.120237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3d6987c8-7270-4948-a758-0f8e43142e9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:26:01.130790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6987c8-7270-4948-a758-0f8e43142e9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:26:01.132390 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6987c8-7270-4948-a758-0f8e43142e9a service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:26:01.141498 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1116/2219] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:26:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:26:11.137837 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:26:20.101201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dba653-a893-4a2b-b9f9-21f8fb822507 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:26:20.112323 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dba653-a893-4a2b-b9f9-21f8fb822507 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:26:20.116183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904758a-c129-4a40-9ae2-a0fb8718c194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:26:20.116874 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904758a-c129-4a40-9ae2-a0fb8718c194 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:26:20.117165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904758a-c129-4a40-9ae2-a0fb8718c194 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:26:20.117442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904758a-c129-4a40-9ae2-a0fb8718c194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:26:20.119467 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9904758a-c129-4a40-9ae2-a0fb8718c194 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:26:20.134471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904758a-c129-4a40-9ae2-a0fb8718c194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:26:20.135979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904758a-c129-4a40-9ae2-a0fb8718c194 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:26:20.148831 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1104/2220] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:26:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:26:30.142791 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=6 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:27:20.767451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6987c8-7270-4948-a758-0f8e43142e9a service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:27:20.769108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3d6987c8-7270-4948-a758-0f8e43142e9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:27:20.793937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6987c8-7270-4948-a758-0f8e43142e9a service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:27:20.797824 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9139f-f5cd-4fc3-8c77-aeb1e61bf669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:27:20.798467 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9139f-f5cd-4fc3-8c77-aeb1e61bf669 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:27:20.798752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9139f-f5cd-4fc3-8c77-aeb1e61bf669 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:27:20.799027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9139f-f5cd-4fc3-8c77-aeb1e61bf669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:27:20.800959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-47b9139f-f5cd-4fc3-8c77-aeb1e61bf669 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:27:20.876020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b9139f-f5cd-4fc3-8c77-aeb1e61bf669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:27:20.877471 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b9139f-f5cd-4fc3-8c77-aeb1e61bf669 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:27:20.887313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1117/2221] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:27:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:27:20.971454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904758a-c129-4a40-9ae2-a0fb8718c194 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:27:20.981715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904758a-c129-4a40-9ae2-a0fb8718c194 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:27:20.986070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fcd1b-16f9-4f2f-931a-cb040837bd71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:27:20.986698 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fcd1b-16f9-4f2f-931a-cb040837bd71 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:27:20.986990 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fcd1b-16f9-4f2f-931a-cb040837bd71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:27:20.987248 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fcd1b-16f9-4f2f-931a-cb040837bd71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:27:20.989123 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-982fcd1b-16f9-4f2f-931a-cb040837bd71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:27:21.062494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fcd1b-16f9-4f2f-931a-cb040837bd71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:27:21.064020 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fcd1b-16f9-4f2f-931a-cb040837bd71 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:27:21.073358 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1105/2222] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:27:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:27:21.162729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9139f-f5cd-4fc3-8c77-aeb1e61bf669 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:27:21.173054 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9139f-f5cd-4fc3-8c77-aeb1e61bf669 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:27:21.177796 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789f110-5913-4dcb-9490-b0715efa1385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:27:21.178606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789f110-5913-4dcb-9490-b0715efa1385 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:27:21.178957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789f110-5913-4dcb-9490-b0715efa1385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:27:21.179302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789f110-5913-4dcb-9490-b0715efa1385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:27:21.181353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4789f110-5913-4dcb-9490-b0715efa1385 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:27:21.298918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789f110-5913-4dcb-9490-b0715efa1385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:27:21.300448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789f110-5913-4dcb-9490-b0715efa1385 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:27:21.322438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1118/2223] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:27:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:27:21.790826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fcd1b-16f9-4f2f-931a-cb040837bd71 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:27:21.793361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-982fcd1b-16f9-4f2f-931a-cb040837bd71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:27:21.861695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789f110-5913-4dcb-9490-b0715efa1385 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:27:21.867531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4789f110-5913-4dcb-9490-b0715efa1385 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:27:21.889913 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789f110-5913-4dcb-9490-b0715efa1385 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:27:21.900141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012cfcc9-295d-4cbc-90d1-6bb771dc44a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:27:21.901038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cfcc9-295d-4cbc-90d1-6bb771dc44a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:27:21.901325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cfcc9-295d-4cbc-90d1-6bb771dc44a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:27:21.901600 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cfcc9-295d-4cbc-90d1-6bb771dc44a9 admin admin] PATH_INFO: `/v3/users/5181b551e8e541c88dcee59ea47d20eb` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:27:21.902852 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012cfcc9-295d-4cbc-90d1-6bb771dc44a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5181b551e8e541c88dcee59ea47d20eb)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:27:21.904916 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012cfcc9-295d-4cbc-90d1-6bb771dc44a9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:27:21.911272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fcd1b-16f9-4f2f-931a-cb040837bd71 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:27:21.916773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f764318-1ee8-4b00-b0dd-0cbf1a6bbf68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:27:21.917460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f764318-1ee8-4b00-b0dd-0cbf1a6bbf68 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:27:21.917904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f764318-1ee8-4b00-b0dd-0cbf1a6bbf68 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:27:21.918196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f764318-1ee8-4b00-b0dd-0cbf1a6bbf68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:27:21.920487 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0f764318-1ee8-4b00-b0dd-0cbf1a6bbf68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:27:21.933749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f764318-1ee8-4b00-b0dd-0cbf1a6bbf68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:27:21.936058 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f764318-1ee8-4b00-b0dd-0cbf1a6bbf68 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:27:21.947250 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1106/2224] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:27:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:27:21.948700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:27:21.959374 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-012cfcc9-295d-4cbc-90d1-6bb771dc44a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5181b551e8e541c88dcee59ea47d20eb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:27:21.960092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-012cfcc9-295d-4cbc-90d1-6bb771dc44a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5181b551e8e541c88dcee59ea47d20eb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:27:21.968625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-012cfcc9-295d-4cbc-90d1-6bb771dc44a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5181b551e8e541c88dcee59ea47d20eb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:27:21.983172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-012cfcc9-295d-4cbc-90d1-6bb771dc44a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5181b551e8e541c88dcee59ea47d20eb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:27:21.990795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-012cfcc9-295d-4cbc-90d1-6bb771dc44a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5181b551e8e541c88dcee59ea47d20eb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:27:21.997035 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1119/2225] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:27:21 2022] DELETE /identity/v3/users/5181b551e8e541c88dcee59ea47d20eb => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:27:22.179123 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f764318-1ee8-4b00-b0dd-0cbf1a6bbf68 service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:27:22.190199 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f764318-1ee8-4b00-b0dd-0cbf1a6bbf68 service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:27:22.251136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124d38c-5c39-4b9e-b573-2871bd2a441f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:27:22.251905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124d38c-5c39-4b9e-b573-2871bd2a441f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:27:22.252213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124d38c-5c39-4b9e-b573-2871bd2a441f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:27:22.252487 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124d38c-5c39-4b9e-b573-2871bd2a441f admin admin] PATH_INFO: `/v3/projects/6a2b0fce04704afa812423f6fee21c84` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:27:22.253656 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124d38c-5c39-4b9e-b573-2871bd2a441f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a2b0fce04704afa812423f6fee21c84)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:27:22.256477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124d38c-5c39-4b9e-b573-2871bd2a441f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:27:22.278770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:27:22.308300 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a124d38c-5c39-4b9e-b573-2871bd2a441f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a2b0fce04704afa812423f6fee21c84', 'request_id': 'req-a124d38c-5c39-4b9e-b573-2871bd2a441f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:27:22.314872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a124d38c-5c39-4b9e-b573-2871bd2a441f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a2b0fce04704afa812423f6fee21c84', 'request_id': 'req-a124d38c-5c39-4b9e-b573-2871bd2a441f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:27:22.323434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a124d38c-5c39-4b9e-b573-2871bd2a441f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a2b0fce04704afa812423f6fee21c84', 'request_id': 'req-a124d38c-5c39-4b9e-b573-2871bd2a441f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:27:22.334820 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a124d38c-5c39-4b9e-b573-2871bd2a441f admin admin] The token cache is being invalidate because project 6a2b0fce04704afa812423f6fee21c84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:27:22.335112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a124d38c-5c39-4b9e-b573-2871bd2a441f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:27:22.335805 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a124d38c-5c39-4b9e-b573-2871bd2a441f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:27:22.337376 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1107/2226] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:27:22 2022] DELETE /identity/v3/projects/6a2b0fce04704afa812423f6fee21c84 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:27:31.993350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=15,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:27:32.331120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:27:35.733208 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012cfcc9-295d-4cbc-90d1-6bb771dc44a9 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:27:35.734773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-012cfcc9-295d-4cbc-90d1-6bb771dc44a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:27:35.793920 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012cfcc9-295d-4cbc-90d1-6bb771dc44a9 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:27:35.846558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2930252-6bc1-4f5e-86db-cf6885cf1aaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:27:35.847205 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2930252-6bc1-4f5e-86db-cf6885cf1aaa service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:27:35.847488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2930252-6bc1-4f5e-86db-cf6885cf1aaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:27:35.847764 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2930252-6bc1-4f5e-86db-cf6885cf1aaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:27:35.850143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a2930252-6bc1-4f5e-86db-cf6885cf1aaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:27:35.921309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2930252-6bc1-4f5e-86db-cf6885cf1aaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:27:35.923012 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2930252-6bc1-4f5e-86db-cf6885cf1aaa service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:27:35.932263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1120/2227] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:27:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:27:45.928102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=73 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:27:57.194038 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124d38c-5c39-4b9e-b573-2871bd2a441f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:27:57.195555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a124d38c-5c39-4b9e-b573-2871bd2a441f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:27:57.207868 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124d38c-5c39-4b9e-b573-2871bd2a441f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:27:57.211676 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079c3e2-5a63-4c99-a4c0-5a184b7bf670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:27:57.212372 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079c3e2-5a63-4c99-a4c0-5a184b7bf670 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:27:57.212661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079c3e2-5a63-4c99-a4c0-5a184b7bf670 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:27:57.212942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079c3e2-5a63-4c99-a4c0-5a184b7bf670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:27:57.214682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f079c3e2-5a63-4c99-a4c0-5a184b7bf670 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:27:57.277301 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079c3e2-5a63-4c99-a4c0-5a184b7bf670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:27:57.278772 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079c3e2-5a63-4c99-a4c0-5a184b7bf670 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:27:57.288486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1108/2228] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:27:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:28:07.283644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=28 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:28:53.384441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2930252-6bc1-4f5e-86db-cf6885cf1aaa service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:28:53.385883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a2930252-6bc1-4f5e-86db-cf6885cf1aaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:28:53.536718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2930252-6bc1-4f5e-86db-cf6885cf1aaa service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:28:53.542616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0c549-3ee7-4d17-83fb-b20dc1459a34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:28:53.543266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0c549-3ee7-4d17-83fb-b20dc1459a34 service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:28:53.543552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0c549-3ee7-4d17-83fb-b20dc1459a34 service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:28:53.543829 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0c549-3ee7-4d17-83fb-b20dc1459a34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:28:53.545681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-01e0c549-3ee7-4d17-83fb-b20dc1459a34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:28:53.558499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0c549-3ee7-4d17-83fb-b20dc1459a34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:28:53.560142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0c549-3ee7-4d17-83fb-b20dc1459a34 service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:28:53.570332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1121/2229] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:28:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:29:03.565915 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=58 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:29:24.850284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079c3e2-5a63-4c99-a4c0-5a184b7bf670 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:29:24.852727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f079c3e2-5a63-4c99-a4c0-5a184b7bf670 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:29:25.024953 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079c3e2-5a63-4c99-a4c0-5a184b7bf670 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:29:25.032776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b32cb-4af7-4a2a-8759-8f6246d63300 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:29:25.033799 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b32cb-4af7-4a2a-8759-8f6246d63300 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:29:25.034569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b32cb-4af7-4a2a-8759-8f6246d63300 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:29:25.035165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b32cb-4af7-4a2a-8759-8f6246d63300 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:29:25.038854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-574b32cb-4af7-4a2a-8759-8f6246d63300 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:29:25.052887 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b32cb-4af7-4a2a-8759-8f6246d63300 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:29:25.054891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b32cb-4af7-4a2a-8759-8f6246d63300 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:29:25.079993 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1109/2230] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:29:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:29:35.067688 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=49 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:30:02.000164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0c549-3ee7-4d17-83fb-b20dc1459a34 service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:30:02.030817 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0c549-3ee7-4d17-83fb-b20dc1459a34 service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:30:02.034580 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873c826-0a4f-4837-bbd9-9cc0471d9437 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:30:02.035228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873c826-0a4f-4837-bbd9-9cc0471d9437 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:30:02.035511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873c826-0a4f-4837-bbd9-9cc0471d9437 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:30:02.035786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873c826-0a4f-4837-bbd9-9cc0471d9437 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:30:02.037618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7873c826-0a4f-4837-bbd9-9cc0471d9437 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:30:02.112712 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873c826-0a4f-4837-bbd9-9cc0471d9437 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:30:02.114161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873c826-0a4f-4837-bbd9-9cc0471d9437 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:30:02.124232 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1122/2231] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:30:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:30:12.120016 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=37 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:30:19.041094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b32cb-4af7-4a2a-8759-8f6246d63300 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:30:19.042628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-574b32cb-4af7-4a2a-8759-8f6246d63300 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:30:19.119782 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b32cb-4af7-4a2a-8759-8f6246d63300 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:30:19.129063 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c152d-3045-4bdd-b788-92d64abc9897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:30:19.129996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c152d-3045-4bdd-b788-92d64abc9897 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:30:19.130281 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c152d-3045-4bdd-b788-92d64abc9897 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:30:19.130564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c152d-3045-4bdd-b788-92d64abc9897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:30:19.132431 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-199c152d-3045-4bdd-b788-92d64abc9897 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:30:19.155828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c152d-3045-4bdd-b788-92d64abc9897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:30:19.157347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c152d-3045-4bdd-b788-92d64abc9897 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:30:19.180912 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1110/2232] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:30:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:30:20.323721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873c826-0a4f-4837-bbd9-9cc0471d9437 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:30:20.337167 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873c826-0a4f-4837-bbd9-9cc0471d9437 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:30:20.342709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de207017-67c1-439b-a261-b425ed22b839 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:30:20.343366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de207017-67c1-439b-a261-b425ed22b839 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:30:20.343653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de207017-67c1-439b-a261-b425ed22b839 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:30:20.343956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de207017-67c1-439b-a261-b425ed22b839 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:30:20.345768 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-de207017-67c1-439b-a261-b425ed22b839 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:30:20.421411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de207017-67c1-439b-a261-b425ed22b839 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:30:20.423287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de207017-67c1-439b-a261-b425ed22b839 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:30:20.432877 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1123/2233] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:30:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:30:29.177597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=37 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:30:30.428398 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=33 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:31:06.049587 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c152d-3045-4bdd-b788-92d64abc9897 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:31:06.051174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-199c152d-3045-4bdd-b788-92d64abc9897 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:31:06.062716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c152d-3045-4bdd-b788-92d64abc9897 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:31:06.066736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b3de3-102e-4d52-a16b-f75e7608a01b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:31:06.067405 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b3de3-102e-4d52-a16b-f75e7608a01b service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:31:06.067690 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b3de3-102e-4d52-a16b-f75e7608a01b service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:31:06.068005 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b3de3-102e-4d52-a16b-f75e7608a01b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:31:06.069761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-142b3de3-102e-4d52-a16b-f75e7608a01b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:31:06.080000 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b3de3-102e-4d52-a16b-f75e7608a01b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:31:06.081444 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b3de3-102e-4d52-a16b-f75e7608a01b service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:31:06.090075 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1111/2234] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:31:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:31:16.085994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=6 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:31:20.616571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de207017-67c1-439b-a261-b425ed22b839 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:31:20.627222 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de207017-67c1-439b-a261-b425ed22b839 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:31:20.631259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e5b2e-8ceb-4a08-93cd-a9d4c5535aa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:31:20.631948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e5b2e-8ceb-4a08-93cd-a9d4c5535aa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:31:20.632262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e5b2e-8ceb-4a08-93cd-a9d4c5535aa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:31:20.632538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e5b2e-8ceb-4a08-93cd-a9d4c5535aa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:31:20.635288 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fb6e5b2e-8ceb-4a08-93cd-a9d4c5535aa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:31:20.650581 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e5b2e-8ceb-4a08-93cd-a9d4c5535aa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:31:20.652093 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e5b2e-8ceb-4a08-93cd-a9d4c5535aa8 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:31:20.664251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1124/2235] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:31:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:31:30.658597 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:33:22.167967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b3de3-102e-4d52-a16b-f75e7608a01b service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:33:22.178492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b3de3-102e-4d52-a16b-f75e7608a01b service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:33:22.182195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f106354-abc5-41c2-92c4-46f89a3d9224 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:33:22.182829 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f106354-abc5-41c2-92c4-46f89a3d9224 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:33:22.183109 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f106354-abc5-41c2-92c4-46f89a3d9224 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:33:22.183382 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f106354-abc5-41c2-92c4-46f89a3d9224 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:33:22.192118 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f106354-abc5-41c2-92c4-46f89a3d9224 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:33:22.193552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f106354-abc5-41c2-92c4-46f89a3d9224 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:33:22.221885 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1112/2236] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:33:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:33:22.269968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e5b2e-8ceb-4a08-93cd-a9d4c5535aa8 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:33:22.280227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e5b2e-8ceb-4a08-93cd-a9d4c5535aa8 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:33:22.283870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0e1fe-7fe3-41e7-b1e9-233912109256 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:33:22.284536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0e1fe-7fe3-41e7-b1e9-233912109256 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:33:22.284823 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0e1fe-7fe3-41e7-b1e9-233912109256 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:33:22.285101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0e1fe-7fe3-41e7-b1e9-233912109256 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:33:22.286850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b6a0e1fe-7fe3-41e7-b1e9-233912109256 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:33:22.356718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0e1fe-7fe3-41e7-b1e9-233912109256 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:33:22.357895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0e1fe-7fe3-41e7-b1e9-233912109256 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:33:22.375674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1125/2237] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:33:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:33:22.422969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f106354-abc5-41c2-92c4-46f89a3d9224 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:33:22.431599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f106354-abc5-41c2-92c4-46f89a3d9224 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:33:22.435891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37845da3-304c-4c87-9fd7-95147dfb8e3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:33:22.436553 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37845da3-304c-4c87-9fd7-95147dfb8e3b service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:33:22.436833 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37845da3-304c-4c87-9fd7-95147dfb8e3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:33:22.437106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37845da3-304c-4c87-9fd7-95147dfb8e3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:33:22.438854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-37845da3-304c-4c87-9fd7-95147dfb8e3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:33:22.511679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37845da3-304c-4c87-9fd7-95147dfb8e3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:33:22.513484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37845da3-304c-4c87-9fd7-95147dfb8e3b service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:33:22.533276 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1113/2238] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:33:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:33:22.578070 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0e1fe-7fe3-41e7-b1e9-233912109256 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:33:22.586498 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0e1fe-7fe3-41e7-b1e9-233912109256 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:33:22.590209 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74d56c-14fc-4d80-a6cd-ffb15d0f13cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:33:22.592038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74d56c-14fc-4d80-a6cd-ffb15d0f13cc service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:33:22.592331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74d56c-14fc-4d80-a6cd-ffb15d0f13cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:33:22.592611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74d56c-14fc-4d80-a6cd-ffb15d0f13cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:33:22.601271 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74d56c-14fc-4d80-a6cd-ffb15d0f13cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:33:22.602713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74d56c-14fc-4d80-a6cd-ffb15d0f13cc service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:33:22.631658 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1126/2239] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:33:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:33:32.530526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=49 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:33:32.629059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=49 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:34:02.366351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37845da3-304c-4c87-9fd7-95147dfb8e3b service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:34:02.378398 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37845da3-304c-4c87-9fd7-95147dfb8e3b service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:34:02.383413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c11f2c-24ff-4265-b9dd-318efa88cda4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:34:02.384082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c11f2c-24ff-4265-b9dd-318efa88cda4 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:34:02.384365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c11f2c-24ff-4265-b9dd-318efa88cda4 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:34:02.384636 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c11f2c-24ff-4265-b9dd-318efa88cda4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:34:02.393609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c11f2c-24ff-4265-b9dd-318efa88cda4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:34:02.395204 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c11f2c-24ff-4265-b9dd-318efa88cda4 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:34:02.405024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1114/2240] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:34:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:34:12.400841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=6 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:35:02.821516 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74d56c-14fc-4d80-a6cd-ffb15d0f13cc service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:02.834241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74d56c-14fc-4d80-a6cd-ffb15d0f13cc service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:02.839673 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2c064-5d9e-458b-bdca-b42e37bc949e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:02.840352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2c064-5d9e-458b-bdca-b42e37bc949e service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:02.840638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2c064-5d9e-458b-bdca-b42e37bc949e service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:02.840916 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2c064-5d9e-458b-bdca-b42e37bc949e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:02.853285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2c064-5d9e-458b-bdca-b42e37bc949e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:02.856214 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2c064-5d9e-458b-bdca-b42e37bc949e service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:02.885307 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1127/2241] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:35:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:35:12.882852 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=13 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:35:19.721823 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c11f2c-24ff-4265-b9dd-318efa88cda4 service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:19.731573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c11f2c-24ff-4265-b9dd-318efa88cda4 service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:19.735108 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b4919-64c0-4b9c-a99e-6ff39a0eca25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:19.735730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b4919-64c0-4b9c-a99e-6ff39a0eca25 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:19.736043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b4919-64c0-4b9c-a99e-6ff39a0eca25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:19.736317 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b4919-64c0-4b9c-a99e-6ff39a0eca25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:19.747329 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b4919-64c0-4b9c-a99e-6ff39a0eca25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:19.748800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b4919-64c0-4b9c-a99e-6ff39a0eca25 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:19.760680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1115/2242] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:35:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:35:20.890285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2c064-5d9e-458b-bdca-b42e37bc949e service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:20.891805 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-05c2c064-5d9e-458b-bdca-b42e37bc949e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:35:20.919048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2c064-5d9e-458b-bdca-b42e37bc949e service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:20.924845 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddcd19-237a-44d4-8159-8a55ece8acd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:20.925487 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddcd19-237a-44d4-8159-8a55ece8acd8 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:20.925774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddcd19-237a-44d4-8159-8a55ece8acd8 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:20.926053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddcd19-237a-44d4-8159-8a55ece8acd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:20.933934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddcd19-237a-44d4-8159-8a55ece8acd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:20.935528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddcd19-237a-44d4-8159-8a55ece8acd8 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:20.964717 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1128/2243] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:35:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:35:23.538978 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b4919-64c0-4b9c-a99e-6ff39a0eca25 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:23.550867 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b4919-64c0-4b9c-a99e-6ff39a0eca25 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:23.555225 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3f13b-c005-4883-a15d-b1488f3975fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:23.556100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3f13b-c005-4883-a15d-b1488f3975fe service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:23.556374 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3f13b-c005-4883-a15d-b1488f3975fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:23.556734 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3f13b-c005-4883-a15d-b1488f3975fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:23.558782 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2be3f13b-c005-4883-a15d-b1488f3975fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:35:23.625502 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3f13b-c005-4883-a15d-b1488f3975fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:23.626954 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3f13b-c005-4883-a15d-b1488f3975fe service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:23.636814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1116/2244] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:35:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:35:25.302082 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddcd19-237a-44d4-8159-8a55ece8acd8 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:25.316454 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddcd19-237a-44d4-8159-8a55ece8acd8 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:25.330910 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283955a-b0f9-4564-b06a-1574cbf09ce2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:25.332581 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283955a-b0f9-4564-b06a-1574cbf09ce2 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:25.333024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283955a-b0f9-4564-b06a-1574cbf09ce2 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:25.333514 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283955a-b0f9-4564-b06a-1574cbf09ce2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:25.356450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283955a-b0f9-4564-b06a-1574cbf09ce2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:25.358921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283955a-b0f9-4564-b06a-1574cbf09ce2 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:25.374745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1129/2245] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:35:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:35:27.413579 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3f13b-c005-4883-a15d-b1488f3975fe service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:27.415600 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2be3f13b-c005-4883-a15d-b1488f3975fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:35:27.435291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3f13b-c005-4883-a15d-b1488f3975fe service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:27.441893 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e37a59-24b2-43dd-9eef-e8ed3aa72c87 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:27.442614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e37a59-24b2-43dd-9eef-e8ed3aa72c87 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:27.442938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e37a59-24b2-43dd-9eef-e8ed3aa72c87 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:27.443218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e37a59-24b2-43dd-9eef-e8ed3aa72c87 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:27.446237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-77e37a59-24b2-43dd-9eef-e8ed3aa72c87 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:35:27.466997 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e37a59-24b2-43dd-9eef-e8ed3aa72c87 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:27.468816 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e37a59-24b2-43dd-9eef-e8ed3aa72c87 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:27.483408 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1117/2246] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:35:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:35:29.784898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283955a-b0f9-4564-b06a-1574cbf09ce2 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:29.794571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283955a-b0f9-4564-b06a-1574cbf09ce2 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:29.798784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570dd43-b748-4807-964a-c17319448b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:29.800000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570dd43-b748-4807-964a-c17319448b88 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:29.800224 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570dd43-b748-4807-964a-c17319448b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:29.800801 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570dd43-b748-4807-964a-c17319448b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:29.802994 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1570dd43-b748-4807-964a-c17319448b88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:35:29.816680 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570dd43-b748-4807-964a-c17319448b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:29.818285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570dd43-b748-4807-964a-c17319448b88 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:29.829374 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1130/2247] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:35:31.320054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e37a59-24b2-43dd-9eef-e8ed3aa72c87 service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:31.335958 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e37a59-24b2-43dd-9eef-e8ed3aa72c87 service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:31.344893 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59761229-54de-4880-9b04-55dfed80f8ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:31.345928 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59761229-54de-4880-9b04-55dfed80f8ba service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:31.347381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59761229-54de-4880-9b04-55dfed80f8ba service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:31.347898 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59761229-54de-4880-9b04-55dfed80f8ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:31.351003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-59761229-54de-4880-9b04-55dfed80f8ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:35:31.368023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59761229-54de-4880-9b04-55dfed80f8ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:31.370570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59761229-54de-4880-9b04-55dfed80f8ba service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:31.390672 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570dd43-b748-4807-964a-c17319448b88 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:31.393267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1570dd43-b748-4807-964a-c17319448b88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:35:31.400518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1118/2248] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:35:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:35:31.408275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570dd43-b748-4807-964a-c17319448b88 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:31.416959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039c55b-8e5c-484b-b431-806499c88795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:31.418505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039c55b-8e5c-484b-b431-806499c88795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:31.418783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039c55b-8e5c-484b-b431-806499c88795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:31.419850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039c55b-8e5c-484b-b431-806499c88795 admin admin] PATH_INFO: `/v3/users/5445e009ab664626a2ab54351b0a9631` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:31.421319 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039c55b-8e5c-484b-b431-806499c88795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5445e009ab664626a2ab54351b0a9631)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:31.424149 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039c55b-8e5c-484b-b431-806499c88795 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:31.492550 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8039c55b-8e5c-484b-b431-806499c88795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5445e009ab664626a2ab54351b0a9631'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:31.494258 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8039c55b-8e5c-484b-b431-806499c88795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5445e009ab664626a2ab54351b0a9631'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:31.516423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8039c55b-8e5c-484b-b431-806499c88795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5445e009ab664626a2ab54351b0a9631'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:31.545106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8039c55b-8e5c-484b-b431-806499c88795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5445e009ab664626a2ab54351b0a9631'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:31.556193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8039c55b-8e5c-484b-b431-806499c88795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5445e009ab664626a2ab54351b0a9631'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:31.564757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1131/2249] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:35:31 2022] DELETE /identity/v3/users/5445e009ab664626a2ab54351b0a9631 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:35:31.883736 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59761229-54de-4880-9b04-55dfed80f8ba service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:31.902960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59761229-54de-4880-9b04-55dfed80f8ba service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:31.985858 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9c989-c2d8-4727-af21-de59531fe9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:31.987693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9c989-c2d8-4727-af21-de59531fe9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:31.988397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9c989-c2d8-4727-af21-de59531fe9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:31.988709 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9c989-c2d8-4727-af21-de59531fe9ce admin admin] PATH_INFO: `/v3/projects/eb15baa22fc949f49f215d65dab0f3b1` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:31.990119 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9c989-c2d8-4727-af21-de59531fe9ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb15baa22fc949f49f215d65dab0f3b1)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:31.992147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9c989-c2d8-4727-af21-de59531fe9ce admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:32.058214 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bfc9c989-c2d8-4727-af21-de59531fe9ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb15baa22fc949f49f215d65dab0f3b1', 'request_id': 'req-bfc9c989-c2d8-4727-af21-de59531fe9ce'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:32.066737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bfc9c989-c2d8-4727-af21-de59531fe9ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb15baa22fc949f49f215d65dab0f3b1', 'request_id': 'req-bfc9c989-c2d8-4727-af21-de59531fe9ce'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:32.070380 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:35:32.077906 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bfc9c989-c2d8-4727-af21-de59531fe9ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb15baa22fc949f49f215d65dab0f3b1', 'request_id': 'req-bfc9c989-c2d8-4727-af21-de59531fe9ce'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:32.094486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bfc9c989-c2d8-4727-af21-de59531fe9ce admin admin] The token cache is being invalidate because project eb15baa22fc949f49f215d65dab0f3b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:35:32.094775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bfc9c989-c2d8-4727-af21-de59531fe9ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:32.097058 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bfc9c989-c2d8-4727-af21-de59531fe9ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:32.100289 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1119/2250] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:35:31 2022] DELETE /identity/v3/projects/eb15baa22fc949f49f215d65dab0f3b1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:35:32.104650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039c55b-8e5c-484b-b431-806499c88795 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:32.107778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8039c55b-8e5c-484b-b431-806499c88795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:35:32.129814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039c55b-8e5c-484b-b431-806499c88795 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:32.198566 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13b0e5-8c30-4b59-aad3-8ae279117f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:32.199612 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13b0e5-8c30-4b59-aad3-8ae279117f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:32.199906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13b0e5-8c30-4b59-aad3-8ae279117f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:32.200212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13b0e5-8c30-4b59-aad3-8ae279117f57 admin admin] PATH_INFO: `/v3/users/83a97e1d803043acadb43266a0bc0cd6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:32.215600 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:35:32.222047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13b0e5-8c30-4b59-aad3-8ae279117f57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83a97e1d803043acadb43266a0bc0cd6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:32.224590 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13b0e5-8c30-4b59-aad3-8ae279117f57 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:32.313963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ae13b0e5-8c30-4b59-aad3-8ae279117f57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83a97e1d803043acadb43266a0bc0cd6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:32.314837 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ae13b0e5-8c30-4b59-aad3-8ae279117f57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83a97e1d803043acadb43266a0bc0cd6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:32.327460 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ae13b0e5-8c30-4b59-aad3-8ae279117f57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83a97e1d803043acadb43266a0bc0cd6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:32.348704 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ae13b0e5-8c30-4b59-aad3-8ae279117f57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83a97e1d803043acadb43266a0bc0cd6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:32.358392 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ae13b0e5-8c30-4b59-aad3-8ae279117f57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83a97e1d803043acadb43266a0bc0cd6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:32.367103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1132/2251] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:35:32 2022] DELETE /identity/v3/users/83a97e1d803043acadb43266a0bc0cd6 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:35:32.794288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9c989-c2d8-4727-af21-de59531fe9ce admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:32.795831 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bfc9c989-c2d8-4727-af21-de59531fe9ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:35:32.812410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9c989-c2d8-4727-af21-de59531fe9ce admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:32.920398 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97269bab-b249-4614-b654-42cabdd6db15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:32.922191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97269bab-b249-4614-b654-42cabdd6db15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:32.922471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97269bab-b249-4614-b654-42cabdd6db15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:32.922957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97269bab-b249-4614-b654-42cabdd6db15 admin admin] PATH_INFO: `/v3/projects/c60bdadef20a4b61b0f6f8db1343d630` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:32.939451 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97269bab-b249-4614-b654-42cabdd6db15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60bdadef20a4b61b0f6f8db1343d630)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:32.942695 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97269bab-b249-4614-b654-42cabdd6db15 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:33.000738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-97269bab-b249-4614-b654-42cabdd6db15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60bdadef20a4b61b0f6f8db1343d630', 'request_id': 'req-97269bab-b249-4614-b654-42cabdd6db15'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:33.008304 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-97269bab-b249-4614-b654-42cabdd6db15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60bdadef20a4b61b0f6f8db1343d630', 'request_id': 'req-97269bab-b249-4614-b654-42cabdd6db15'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:33.016194 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-97269bab-b249-4614-b654-42cabdd6db15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60bdadef20a4b61b0f6f8db1343d630', 'request_id': 'req-97269bab-b249-4614-b654-42cabdd6db15'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:33.028237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-97269bab-b249-4614-b654-42cabdd6db15 admin admin] The token cache is being invalidate because project c60bdadef20a4b61b0f6f8db1343d630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:35:33.028520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-97269bab-b249-4614-b654-42cabdd6db15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:33.029209 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-97269bab-b249-4614-b654-42cabdd6db15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:33.030755 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1120/2252] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:35:32 2022] DELETE /identity/v3/projects/c60bdadef20a4b61b0f6f8db1343d630 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:35:33.814681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13b0e5-8c30-4b59-aad3-8ae279117f57 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:33.816726 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ae13b0e5-8c30-4b59-aad3-8ae279117f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:35:33.830419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13b0e5-8c30-4b59-aad3-8ae279117f57 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:33.889156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7a8cf-dae7-4d59-9cb0-bf216cadd4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:33.890086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7a8cf-dae7-4d59-9cb0-bf216cadd4b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:33.890321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7a8cf-dae7-4d59-9cb0-bf216cadd4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:33.890597 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7a8cf-dae7-4d59-9cb0-bf216cadd4b4 admin admin] PATH_INFO: `/v3/users/be1372c20ad94145b775663a9037d55a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:33.891815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7a8cf-dae7-4d59-9cb0-bf216cadd4b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be1372c20ad94145b775663a9037d55a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:33.893826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7a8cf-dae7-4d59-9cb0-bf216cadd4b4 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:33.942733 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c4f7a8cf-dae7-4d59-9cb0-bf216cadd4b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be1372c20ad94145b775663a9037d55a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:33.946334 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c4f7a8cf-dae7-4d59-9cb0-bf216cadd4b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be1372c20ad94145b775663a9037d55a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:33.954566 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c4f7a8cf-dae7-4d59-9cb0-bf216cadd4b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be1372c20ad94145b775663a9037d55a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:33.968651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c4f7a8cf-dae7-4d59-9cb0-bf216cadd4b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be1372c20ad94145b775663a9037d55a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:33.975042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:35:33.978652 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c4f7a8cf-dae7-4d59-9cb0-bf216cadd4b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be1372c20ad94145b775663a9037d55a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:33.989930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1133/2253] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:35:33 2022] DELETE /identity/v3/users/be1372c20ad94145b775663a9037d55a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:35:34.191336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97269bab-b249-4614-b654-42cabdd6db15 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:34.192854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-97269bab-b249-4614-b654-42cabdd6db15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:35:34.206351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97269bab-b249-4614-b654-42cabdd6db15 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:34.267614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09954f6e-2c68-4b83-9977-ebcb58864d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:34.269128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09954f6e-2c68-4b83-9977-ebcb58864d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:34.269410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09954f6e-2c68-4b83-9977-ebcb58864d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:34.269740 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09954f6e-2c68-4b83-9977-ebcb58864d2a admin admin] PATH_INFO: `/v3/projects/7efd9ffb80ea497b93cc2e499bda86f7` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:34.270990 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09954f6e-2c68-4b83-9977-ebcb58864d2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7efd9ffb80ea497b93cc2e499bda86f7)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:34.272969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09954f6e-2c68-4b83-9977-ebcb58864d2a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:34.282121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:35:34.326382 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-09954f6e-2c68-4b83-9977-ebcb58864d2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7efd9ffb80ea497b93cc2e499bda86f7', 'request_id': 'req-09954f6e-2c68-4b83-9977-ebcb58864d2a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:34.333198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-09954f6e-2c68-4b83-9977-ebcb58864d2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7efd9ffb80ea497b93cc2e499bda86f7', 'request_id': 'req-09954f6e-2c68-4b83-9977-ebcb58864d2a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:34.341450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-09954f6e-2c68-4b83-9977-ebcb58864d2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7efd9ffb80ea497b93cc2e499bda86f7', 'request_id': 'req-09954f6e-2c68-4b83-9977-ebcb58864d2a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:34.353605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-09954f6e-2c68-4b83-9977-ebcb58864d2a admin admin] The token cache is being invalidate because project 7efd9ffb80ea497b93cc2e499bda86f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:35:34.353896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-09954f6e-2c68-4b83-9977-ebcb58864d2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:34.354548 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-09954f6e-2c68-4b83-9977-ebcb58864d2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:35:34.356152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1121/2254] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:35:34 2022] DELETE /identity/v3/projects/7efd9ffb80ea497b93cc2e499bda86f7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:35:34.362176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b9dcb-77d2-4255-bef5-a27c500a8b43 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:34.362464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b9dcb-77d2-4255-bef5-a27c500a8b43 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:34.362836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b9dcb-77d2-4255-bef5-a27c500a8b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:34.427087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f43b9dcb-77d2-4255-bef5-a27c500a8b43 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:35:34.428414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f43b9dcb-77d2-4255-bef5-a27c500a8b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:35:34.494849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1134/2255] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:35:34.504061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3751cf-1c3e-4a77-b284-08f6968b3bbc None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:34.504350 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3751cf-1c3e-4a77-b284-08f6968b3bbc None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:34.504628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3751cf-1c3e-4a77-b284-08f6968b3bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:34.527173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6c3751cf-1c3e-4a77-b284-08f6968b3bbc None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:35:34.528512 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6c3751cf-1c3e-4a77-b284-08f6968b3bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:35:34.537848 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1122/2256] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:35:34.542354 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b9dcb-77d2-4255-bef5-a27c500a8b43 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:34.553871 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b9dcb-77d2-4255-bef5-a27c500a8b43 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:34.561723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20f781-0145-4e3f-bfd5-515245d44fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:34.562431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20f781-0145-4e3f-bfd5-515245d44fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:34.562710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20f781-0145-4e3f-bfd5-515245d44fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:34.562979 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20f781-0145-4e3f-bfd5-515245d44fa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:34.563710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20f781-0145-4e3f-bfd5-515245d44fa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:35:34.564481 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20f781-0145-4e3f-bfd5-515245d44fa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:34.566092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20f781-0145-4e3f-bfd5-515245d44fa7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:34.578464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1135/2257] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:35:34.590228 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3751cf-1c3e-4a77-b284-08f6968b3bbc None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:34.601193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3751cf-1c3e-4a77-b284-08f6968b3bbc None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:34.605986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c6fd7-605f-4a15-b8a1-74c5901c6c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:34.606643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c6fd7-605f-4a15-b8a1-74c5901c6c52 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:34.606940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c6fd7-605f-4a15-b8a1-74c5901c6c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:34.607201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c6fd7-605f-4a15-b8a1-74c5901c6c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:34.607951 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c6fd7-605f-4a15-b8a1-74c5901c6c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:34.609837 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c6fd7-605f-4a15-b8a1-74c5901c6c52 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:34.647564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1123/2258] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:35:34 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:35:34.652045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20f781-0145-4e3f-bfd5-515245d44fa7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:34.663542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20f781-0145-4e3f-bfd5-515245d44fa7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:34.725532 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eff7d9-37a0-4751-81d8-3b2b3d14634d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:34.726225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eff7d9-37a0-4751-81d8-3b2b3d14634d admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:34.726518 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eff7d9-37a0-4751-81d8-3b2b3d14634d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:34.726794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eff7d9-37a0-4751-81d8-3b2b3d14634d admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:34.727527 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eff7d9-37a0-4751-81d8-3b2b3d14634d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:34.729594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eff7d9-37a0-4751-81d8-3b2b3d14634d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:34.752075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1136/2259] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:35:34 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:35:34.756809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c6fd7-605f-4a15-b8a1-74c5901c6c52 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:34.767628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c6fd7-605f-4a15-b8a1-74c5901c6c52 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:34.773975 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ce609-0e85-4a9b-88c6-a0c0a7c4368a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:34.774834 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ce609-0e85-4a9b-88c6-a0c0a7c4368a admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:34.775173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ce609-0e85-4a9b-88c6-a0c0a7c4368a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:34.775449 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ce609-0e85-4a9b-88c6-a0c0a7c4368a admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:34.776263 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ce609-0e85-4a9b-88c6-a0c0a7c4368a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:34.778193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ce609-0e85-4a9b-88c6-a0c0a7c4368a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:34.791415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1124/2260] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:35:34.795661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eff7d9-37a0-4751-81d8-3b2b3d14634d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:34.806580 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eff7d9-37a0-4751-81d8-3b2b3d14634d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:34.811341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76fc4d-a32f-4ca7-9a5e-1b4cda0707e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:34.812067 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76fc4d-a32f-4ca7-9a5e-1b4cda0707e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:34.812356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76fc4d-a32f-4ca7-9a5e-1b4cda0707e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:34.812634 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76fc4d-a32f-4ca7-9a5e-1b4cda0707e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:34.813407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76fc4d-a32f-4ca7-9a5e-1b4cda0707e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:34.816696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76fc4d-a32f-4ca7-9a5e-1b4cda0707e0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:34.827593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1137/2261] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:35:34.832878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ce609-0e85-4a9b-88c6-a0c0a7c4368a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:35:34.845515 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ce609-0e85-4a9b-88c6-a0c0a7c4368a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:35:34.850529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53273d11-555a-4abf-9e3b-9e81d0e4f74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:35:34.851423 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53273d11-555a-4abf-9e3b-9e81d0e4f74b admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:34.851705 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53273d11-555a-4abf-9e3b-9e81d0e4f74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:34.852021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53273d11-555a-4abf-9e3b-9e81d0e4f74b admin admin] PATH_INFO: `/v3/projects/955f8dfdc5244891942baa392bc2e99a/users/3282c6ce409b43218c6c28315c23bb97/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:34.868013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53273d11-555a-4abf-9e3b-9e81d0e4f74b admin admin] RBAC: Authorizing `identity:create_grant(project_id=955f8dfdc5244891942baa392bc2e99a, user_id=3282c6ce409b43218c6c28315c23bb97, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:35:34.870750 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53273d11-555a-4abf-9e3b-9e81d0e4f74b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:35:34.886126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1125/2262] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:35:34 2022] PUT /identity/v3/projects/955f8dfdc5244891942baa392bc2e99a/users/3282c6ce409b43218c6c28315c23bb97/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:35:34.897833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487aa164-849e-4589-b73c-fc3f9d70bb9f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:35:34.898127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487aa164-849e-4589-b73c-fc3f9d70bb9f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:35:34.898438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487aa164-849e-4589-b73c-fc3f9d70bb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:35:34.928063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-487aa164-849e-4589-b73c-fc3f9d70bb9f None None] MFA Rules not processed for user `3282c6ce409b43218c6c28315c23bb97`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:35:34.929300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-487aa164-849e-4589-b73c-fc3f9d70bb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:35:34.947505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:35:34.997886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1138/2263] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:35:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:35:44.881960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:35:44.994772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=19 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:36:21.673816 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53273d11-555a-4abf-9e3b-9e81d0e4f74b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:21.675572 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-53273d11-555a-4abf-9e3b-9e81d0e4f74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:36:21.761472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53273d11-555a-4abf-9e3b-9e81d0e4f74b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:21.765181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06118c3-687e-449e-85f9-0b1ce1da1f7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:21.765751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06118c3-687e-449e-85f9-0b1ce1da1f7b service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:21.766033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06118c3-687e-449e-85f9-0b1ce1da1f7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:21.766304 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06118c3-687e-449e-85f9-0b1ce1da1f7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:21.768883 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e06118c3-687e-449e-85f9-0b1ce1da1f7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:36:21.847827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06118c3-687e-449e-85f9-0b1ce1da1f7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:21.849412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06118c3-687e-449e-85f9-0b1ce1da1f7b service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:21.862499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1126/2264] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:36:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:36:31.856361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=59 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:36:35.065140 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487aa164-849e-4589-b73c-fc3f9d70bb9f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:35.132909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487aa164-849e-4589-b73c-fc3f9d70bb9f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:35.137629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e798cb-b531-4642-b933-614365e19d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:35.138540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e798cb-b531-4642-b933-614365e19d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:35.138911 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e798cb-b531-4642-b933-614365e19d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:35.139094 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e798cb-b531-4642-b933-614365e19d08 admin admin] PATH_INFO: `/v3/users/3282c6ce409b43218c6c28315c23bb97` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:35.140342 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e798cb-b531-4642-b933-614365e19d08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3282c6ce409b43218c6c28315c23bb97)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:35.143062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e798cb-b531-4642-b933-614365e19d08 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:35.193580 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a1e798cb-b531-4642-b933-614365e19d08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3282c6ce409b43218c6c28315c23bb97'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:36:35.194258 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a1e798cb-b531-4642-b933-614365e19d08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3282c6ce409b43218c6c28315c23bb97'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:36:35.202891 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a1e798cb-b531-4642-b933-614365e19d08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3282c6ce409b43218c6c28315c23bb97'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:36:35.217859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a1e798cb-b531-4642-b933-614365e19d08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3282c6ce409b43218c6c28315c23bb97'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:36:35.227752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a1e798cb-b531-4642-b933-614365e19d08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3282c6ce409b43218c6c28315c23bb97'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:36:35.233835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1139/2265] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:36:35 2022] DELETE /identity/v3/users/3282c6ce409b43218c6c28315c23bb97 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:36:35.238977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d801e-1c85-4a7b-a938-b25b6cd4d9ef None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:35.239469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d801e-1c85-4a7b-a938-b25b6cd4d9ef None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:35.239590 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d801e-1c85-4a7b-a938-b25b6cd4d9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:35.257439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a64d801e-1c85-4a7b-a938-b25b6cd4d9ef None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:36:35.258646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a64d801e-1c85-4a7b-a938-b25b6cd4d9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:36:35.324133 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1127/2266] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:36:35.334789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e798cb-b531-4642-b933-614365e19d08 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:35.336335 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a1e798cb-b531-4642-b933-614365e19d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:36:35.403030 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e798cb-b531-4642-b933-614365e19d08 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:35.407087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ff840-7b81-45e8-b017-c79ed04701d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:35.408293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ff840-7b81-45e8-b017-c79ed04701d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:35.408953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ff840-7b81-45e8-b017-c79ed04701d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:35.408953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ff840-7b81-45e8-b017-c79ed04701d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:35.418881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ff840-7b81-45e8-b017-c79ed04701d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:35.420369 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ff840-7b81-45e8-b017-c79ed04701d8 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:35.431173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1140/2267] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:36:35.729420 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d801e-1c85-4a7b-a938-b25b6cd4d9ef None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:35.730845 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a64d801e-1c85-4a7b-a938-b25b6cd4d9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:36:35.746301 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d801e-1c85-4a7b-a938-b25b6cd4d9ef None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:35.750160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e725348-34c7-4214-a820-e4e316f4184b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:35.750547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e725348-34c7-4214-a820-e4e316f4184b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:35.750831 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e725348-34c7-4214-a820-e4e316f4184b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:35.751127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e725348-34c7-4214-a820-e4e316f4184b admin admin] PATH_INFO: `/v3/projects/955f8dfdc5244891942baa392bc2e99a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:35.752452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e725348-34c7-4214-a820-e4e316f4184b admin admin] RBAC: Authorizing `identity:delete_project(project_id=955f8dfdc5244891942baa392bc2e99a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:35.754414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e725348-34c7-4214-a820-e4e316f4184b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:35.813753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6e725348-34c7-4214-a820-e4e316f4184b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '955f8dfdc5244891942baa392bc2e99a', 'request_id': 'req-6e725348-34c7-4214-a820-e4e316f4184b'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:36:35.822271 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6e725348-34c7-4214-a820-e4e316f4184b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '955f8dfdc5244891942baa392bc2e99a', 'request_id': 'req-6e725348-34c7-4214-a820-e4e316f4184b'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:36:35.831715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6e725348-34c7-4214-a820-e4e316f4184b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '955f8dfdc5244891942baa392bc2e99a', 'request_id': 'req-6e725348-34c7-4214-a820-e4e316f4184b'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:36:35.842760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6e725348-34c7-4214-a820-e4e316f4184b admin admin] The token cache is being invalidate because project 955f8dfdc5244891942baa392bc2e99a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:36:35.843069 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6e725348-34c7-4214-a820-e4e316f4184b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:36:35.843716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6e725348-34c7-4214-a820-e4e316f4184b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:36:35.846491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1128/2268] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:36:35 2022] DELETE /identity/v3/projects/955f8dfdc5244891942baa392bc2e99a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:36:35.859791 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6693a8-00cc-4e43-ae62-13701cab8783 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:35.860098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6693a8-00cc-4e43-ae62-13701cab8783 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:35.860381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6693a8-00cc-4e43-ae62-13701cab8783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:35.878558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-fc6693a8-00cc-4e43-ae62-13701cab8783 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:36:35.879785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fc6693a8-00cc-4e43-ae62-13701cab8783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:36:35.897863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:36:35.946920 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1141/2269] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:36:35.956176 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d45be-b13c-4a83-8955-1673a1dbc572 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:35.956468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d45be-b13c-4a83-8955-1673a1dbc572 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:35.956743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d45be-b13c-4a83-8955-1673a1dbc572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:35.976857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-178d45be-b13c-4a83-8955-1673a1dbc572 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:36:35.978083 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-178d45be-b13c-4a83-8955-1673a1dbc572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:36:35.987726 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1129/2270] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:36:35.992106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6693a8-00cc-4e43-ae62-13701cab8783 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:36.002337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6693a8-00cc-4e43-ae62-13701cab8783 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:36.007049 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0d9bb-a534-4178-946a-14a0e749379a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:36.007715 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0d9bb-a534-4178-946a-14a0e749379a admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:36.008037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0d9bb-a534-4178-946a-14a0e749379a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:36.008318 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0d9bb-a534-4178-946a-14a0e749379a admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:36.009057 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0d9bb-a534-4178-946a-14a0e749379a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:36:36.009360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0d9bb-a534-4178-946a-14a0e749379a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:36.010994 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0d9bb-a534-4178-946a-14a0e749379a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:36.024813 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1142/2271] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:36:36.029476 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d45be-b13c-4a83-8955-1673a1dbc572 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:36.041253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d45be-b13c-4a83-8955-1673a1dbc572 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:36.047742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415bbf1-d39d-40a9-b252-5468ee7cb037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:36.048554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415bbf1-d39d-40a9-b252-5468ee7cb037 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:36.048832 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415bbf1-d39d-40a9-b252-5468ee7cb037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:36.049099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415bbf1-d39d-40a9-b252-5468ee7cb037 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:36.049815 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415bbf1-d39d-40a9-b252-5468ee7cb037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:36.052166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415bbf1-d39d-40a9-b252-5468ee7cb037 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:36.081478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1130/2272] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:36:36 2022] POST /identity/v3/projects => generated 357 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:36:36.085950 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0d9bb-a534-4178-946a-14a0e749379a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:36.097206 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0d9bb-a534-4178-946a-14a0e749379a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:36.160386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134945a-2f11-4449-999e-b5dcfb0b1931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:36.161040 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134945a-2f11-4449-999e-b5dcfb0b1931 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:36.161404 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134945a-2f11-4449-999e-b5dcfb0b1931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:36.161637 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134945a-2f11-4449-999e-b5dcfb0b1931 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:36.162337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134945a-2f11-4449-999e-b5dcfb0b1931 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:36.164283 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134945a-2f11-4449-999e-b5dcfb0b1931 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:36.188380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1143/2273] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:36:36 2022] POST /identity/v3/users => generated 279 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:36:36.193130 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415bbf1-d39d-40a9-b252-5468ee7cb037 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:36.205018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415bbf1-d39d-40a9-b252-5468ee7cb037 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:36.211100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687a66d-3a1a-4636-bbf6-2e6144ebfef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:36.211820 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687a66d-3a1a-4636-bbf6-2e6144ebfef6 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:36.212144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687a66d-3a1a-4636-bbf6-2e6144ebfef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:36.212417 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687a66d-3a1a-4636-bbf6-2e6144ebfef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:36.213161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687a66d-3a1a-4636-bbf6-2e6144ebfef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:36.215007 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687a66d-3a1a-4636-bbf6-2e6144ebfef6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:36.226596 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1131/2274] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:36:36.230617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134945a-2f11-4449-999e-b5dcfb0b1931 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:36.242016 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134945a-2f11-4449-999e-b5dcfb0b1931 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:36.246929 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf5c00-12ba-44d9-89c1-f5b650f89d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:36.247610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf5c00-12ba-44d9-89c1-f5b650f89d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:36.248077 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf5c00-12ba-44d9-89c1-f5b650f89d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:36.248360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf5c00-12ba-44d9-89c1-f5b650f89d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:36.249139 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf5c00-12ba-44d9-89c1-f5b650f89d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:36.250757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf5c00-12ba-44d9-89c1-f5b650f89d61 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:36.262088 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1144/2275] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:36:36.267056 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687a66d-3a1a-4636-bbf6-2e6144ebfef6 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:36.279539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687a66d-3a1a-4636-bbf6-2e6144ebfef6 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:36.288964 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aea4e8-2cd2-4d46-ba8a-0cce9067b840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:36.289882 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aea4e8-2cd2-4d46-ba8a-0cce9067b840 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:36.290168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aea4e8-2cd2-4d46-ba8a-0cce9067b840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:36.290446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aea4e8-2cd2-4d46-ba8a-0cce9067b840 admin admin] PATH_INFO: `/v3/projects/65b65d9f24b442d0aebce4ee10022176/users/113b1059439a4cf98ba357053425830c/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:36.308144 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aea4e8-2cd2-4d46-ba8a-0cce9067b840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b65d9f24b442d0aebce4ee10022176, user_id=113b1059439a4cf98ba357053425830c, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:36.310880 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aea4e8-2cd2-4d46-ba8a-0cce9067b840 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:36.330593 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1132/2276] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:36:36 2022] PUT /identity/v3/projects/65b65d9f24b442d0aebce4ee10022176/users/113b1059439a4cf98ba357053425830c/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:36:36.346829 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94cdca-3277-48ae-8128-6da8d69d329d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:36.347270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94cdca-3277-48ae-8128-6da8d69d329d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:36.347668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94cdca-3277-48ae-8128-6da8d69d329d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:36.385851 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-5a94cdca-3277-48ae-8128-6da8d69d329d None None] MFA Rules not processed for user `113b1059439a4cf98ba357053425830c`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:36:36.387095 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5a94cdca-3277-48ae-8128-6da8d69d329d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:36:36.456357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1145/2277] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:36:36 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:36:36.461051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aea4e8-2cd2-4d46-ba8a-0cce9067b840 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:36.483846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:36:36.531776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aea4e8-2cd2-4d46-ba8a-0cce9067b840 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:36.536591 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd640f5-37d1-4c07-a3f0-ca1cd6d459ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:36.537249 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd640f5-37d1-4c07-a3f0-ca1cd6d459ef admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:36.537538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd640f5-37d1-4c07-a3f0-ca1cd6d459ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:36.537818 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd640f5-37d1-4c07-a3f0-ca1cd6d459ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:36.538858 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd640f5-37d1-4c07-a3f0-ca1cd6d459ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:36.540906 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd640f5-37d1-4c07-a3f0-ca1cd6d459ef admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:36.573018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1133/2278] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:36:36 2022] POST /identity/v3/projects => generated 357 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:36:36.577236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a94cdca-3277-48ae-8128-6da8d69d329d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:36.593292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a94cdca-3277-48ae-8128-6da8d69d329d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:36.607061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:36:36.656447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfc538-1bb6-47f4-9632-83bfb1d6b6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:36.657415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfc538-1bb6-47f4-9632-83bfb1d6b6a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:36.657674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfc538-1bb6-47f4-9632-83bfb1d6b6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:36.658184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfc538-1bb6-47f4-9632-83bfb1d6b6a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:36.659082 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfc538-1bb6-47f4-9632-83bfb1d6b6a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:36.661573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfc538-1bb6-47f4-9632-83bfb1d6b6a5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:36.683996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1146/2279] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:36:36 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:36:36.688606 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd640f5-37d1-4c07-a3f0-ca1cd6d459ef admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:36.700586 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd640f5-37d1-4c07-a3f0-ca1cd6d459ef admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:36.705471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca35703-3964-4d78-a0f8-953c24ba69b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:36.706522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca35703-3964-4d78-a0f8-953c24ba69b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:36.706814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca35703-3964-4d78-a0f8-953c24ba69b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:36.707094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca35703-3964-4d78-a0f8-953c24ba69b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:36.707861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca35703-3964-4d78-a0f8-953c24ba69b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:36.709601 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca35703-3964-4d78-a0f8-953c24ba69b0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:36.723962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1134/2280] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:36:36.727415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfc538-1bb6-47f4-9632-83bfb1d6b6a5 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:36.739072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfc538-1bb6-47f4-9632-83bfb1d6b6a5 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:36.743869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3bb3a-2d45-450f-8b91-2d197e93723d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:36.744802 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3bb3a-2d45-450f-8b91-2d197e93723d admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:36.745087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3bb3a-2d45-450f-8b91-2d197e93723d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:36.745363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3bb3a-2d45-450f-8b91-2d197e93723d admin admin] PATH_INFO: `/v3/projects/731597dafaf44528a7c44343e2a961fd/users/9d610a4ba2764e6692530c6fca30e4e9/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:36.762229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3bb3a-2d45-450f-8b91-2d197e93723d admin admin] RBAC: Authorizing `identity:create_grant(project_id=731597dafaf44528a7c44343e2a961fd, user_id=9d610a4ba2764e6692530c6fca30e4e9, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:36.765002 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3bb3a-2d45-450f-8b91-2d197e93723d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:36.780992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1147/2281] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:36:36 2022] PUT /identity/v3/projects/731597dafaf44528a7c44343e2a961fd/users/9d610a4ba2764e6692530c6fca30e4e9/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:36:36.785732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca35703-3964-4d78-a0f8-953c24ba69b0 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:36.796814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca35703-3964-4d78-a0f8-953c24ba69b0 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:36.856292 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bd5a3-6b14-4298-945f-08ef1bc2116c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:36.856869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bd5a3-6b14-4298-945f-08ef1bc2116c admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:36.857149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bd5a3-6b14-4298-945f-08ef1bc2116c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:36.857421 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bd5a3-6b14-4298-945f-08ef1bc2116c admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:36.858148 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bd5a3-6b14-4298-945f-08ef1bc2116c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:36.859744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bd5a3-6b14-4298-945f-08ef1bc2116c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:36.870581 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1135/2282] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:36:36.874895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3bb3a-2d45-450f-8b91-2d197e93723d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:36:36.886163 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3bb3a-2d45-450f-8b91-2d197e93723d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:36:36.890794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fca443-e6d6-4ff8-afee-182710c55896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:36:36.891666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fca443-e6d6-4ff8-afee-182710c55896 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:36.891993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fca443-e6d6-4ff8-afee-182710c55896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:36.892277 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fca443-e6d6-4ff8-afee-182710c55896 admin admin] PATH_INFO: `/v3/projects/731597dafaf44528a7c44343e2a961fd/users/9d610a4ba2764e6692530c6fca30e4e9/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:36.894359 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fca443-e6d6-4ff8-afee-182710c55896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=731597dafaf44528a7c44343e2a961fd, user_id=9d610a4ba2764e6692530c6fca30e4e9, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:36:36.897087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fca443-e6d6-4ff8-afee-182710c55896 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:36:36.907291 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1148/2283] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:36:36 2022] PUT /identity/v3/projects/731597dafaf44528a7c44343e2a961fd/users/9d610a4ba2764e6692530c6fca30e4e9/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:36:36.926394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce503f5e-362d-4f62-8ea6-1fa86a7c9755 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:36:36.926680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce503f5e-362d-4f62-8ea6-1fa86a7c9755 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:36:36.926964 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce503f5e-362d-4f62-8ea6-1fa86a7c9755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:36:36.953725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ce503f5e-362d-4f62-8ea6-1fa86a7c9755 None None] MFA Rules not processed for user `9d610a4ba2764e6692530c6fca30e4e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:36:36.955062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ce503f5e-362d-4f62-8ea6-1fa86a7c9755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:36:37.022350 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1136/2284] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:36:36 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:36:46.903554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=36,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:36:47.019477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:37:22.953855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09f4c4-f359-45a5-b538-6447ea081d28 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:37:22.954552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09f4c4-f359-45a5-b538-6447ea081d28 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:37:22.954552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09f4c4-f359-45a5-b538-6447ea081d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:37:23.001078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6b09f4c4-f359-45a5-b538-6447ea081d28 None None] MFA Rules not processed for user `7b4006c43f7944c083cf323db4df2b3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:37:23.002435 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6b09f4c4-f359-45a5-b538-6447ea081d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:37:23.082494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1149/2285] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:37:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:37:23.478268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce503f5e-362d-4f62-8ea6-1fa86a7c9755 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:37:23.480107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ce503f5e-362d-4f62-8ea6-1fa86a7c9755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:37:23.561686 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09f4c4-f359-45a5-b538-6447ea081d28 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:37:23.564094 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6b09f4c4-f359-45a5-b538-6447ea081d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:37:23.573953 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce503f5e-362d-4f62-8ea6-1fa86a7c9755 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:37:23.580117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a65cb-917f-4eb5-93fd-b639c3f59c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:37:23.581133 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a65cb-917f-4eb5-93fd-b639c3f59c6c service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:37:23.581657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a65cb-917f-4eb5-93fd-b639c3f59c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:37:23.582136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a65cb-917f-4eb5-93fd-b639c3f59c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:37:23.585457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-157a65cb-917f-4eb5-93fd-b639c3f59c6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:37:23.602731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a65cb-917f-4eb5-93fd-b639c3f59c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:37:23.605052 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a65cb-917f-4eb5-93fd-b639c3f59c6c service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:37:23.619227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1137/2286] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:37:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:37:23.699288 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09f4c4-f359-45a5-b538-6447ea081d28 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:37:23.706311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b417d-193c-42d3-a30f-c932e61137d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:37:23.707800 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b417d-193c-42d3-a30f-c932e61137d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:37:23.709236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b417d-193c-42d3-a30f-c932e61137d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:37:23.709849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b417d-193c-42d3-a30f-c932e61137d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:37:23.718280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-642b417d-193c-42d3-a30f-c932e61137d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:37:23.743447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-642b417d-193c-42d3-a30f-c932e61137d8 service cinder] Could not find project: 955f8dfdc5244891942baa392bc2e99a.: keystone.exception.ProjectNotFound: Could not find project: 955f8dfdc5244891942baa392bc2e99a. Jul 21 09:37:23.743447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 09:37:23.743447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 09:37:23.743447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 09:37:23.743447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 09:37:23.743447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 09:37:23.743447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 09:37:23.743447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 09:37:23.743447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 09:37:23.743447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 09:37:23.743447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 09:37:23.743447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 09:37:23.743447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 21 09:37:23.743447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 21 09:37:23.743447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 09:37:23.745885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 09:37:23.750100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 09:37:23.753776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 09:37:23.756113 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 955f8dfdc5244891942baa392bc2e99a. Jul 21 09:37:23.756113 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 21 09:37:23.756113 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1150/2287] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:37:23 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 21 09:37:23.784929 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a65cb-917f-4eb5-93fd-b639c3f59c6c service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:37:23.787415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-157a65cb-917f-4eb5-93fd-b639c3f59c6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:37:23.817115 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a65cb-917f-4eb5-93fd-b639c3f59c6c service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:37:23.834876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fbf49-d943-46fb-9b74-926045f92ec3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:37:23.839844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fbf49-d943-46fb-9b74-926045f92ec3 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:37:23.839844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fbf49-d943-46fb-9b74-926045f92ec3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:37:23.839844 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fbf49-d943-46fb-9b74-926045f92ec3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:37:23.868250 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fbf49-d943-46fb-9b74-926045f92ec3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:37:23.871086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fbf49-d943-46fb-9b74-926045f92ec3 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:37:23.892863 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1138/2288] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:37:23 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:37:24.142556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b417d-193c-42d3-a30f-c932e61137d8 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:37:24.144629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-642b417d-193c-42d3-a30f-c932e61137d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:37:24.274659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b417d-193c-42d3-a30f-c932e61137d8 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:37:24.280112 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb91d80-75e7-4865-acbe-7ecb137d41f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:37:24.280683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb91d80-75e7-4865-acbe-7ecb137d41f2 service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:37:24.280969 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb91d80-75e7-4865-acbe-7ecb137d41f2 service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:37:24.281248 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb91d80-75e7-4865-acbe-7ecb137d41f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:37:24.283414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cdb91d80-75e7-4865-acbe-7ecb137d41f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:37:24.296054 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb91d80-75e7-4865-acbe-7ecb137d41f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:37:24.298025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb91d80-75e7-4865-acbe-7ecb137d41f2 service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:37:24.310067 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1151/2289] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:37:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:37:26.310150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fbf49-d943-46fb-9b74-926045f92ec3 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:37:26.311792 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0f7fbf49-d943-46fb-9b74-926045f92ec3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:37:26.329883 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fbf49-d943-46fb-9b74-926045f92ec3 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:37:26.336739 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff33da-eb50-44fd-af48-9e84f78cc8af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:37:26.337717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff33da-eb50-44fd-af48-9e84f78cc8af service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:37:26.338125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff33da-eb50-44fd-af48-9e84f78cc8af service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:37:26.338407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff33da-eb50-44fd-af48-9e84f78cc8af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:37:26.340361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ffff33da-eb50-44fd-af48-9e84f78cc8af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:37:26.352816 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff33da-eb50-44fd-af48-9e84f78cc8af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:37:26.354596 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff33da-eb50-44fd-af48-9e84f78cc8af service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:37:26.364751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1139/2290] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:37:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:37:33.792457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb91d80-75e7-4865-acbe-7ecb137d41f2 service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:37:33.793860 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cdb91d80-75e7-4865-acbe-7ecb137d41f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:37:33.810384 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:37:33.873894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb91d80-75e7-4865-acbe-7ecb137d41f2 service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:37:33.880468 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3e088-94c9-45f7-be76-94237682184f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:37:33.880888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3e088-94c9-45f7-be76-94237682184f service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:37:33.881175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3e088-94c9-45f7-be76-94237682184f service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:37:33.881541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3e088-94c9-45f7-be76-94237682184f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:37:33.883211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d8c3e088-94c9-45f7-be76-94237682184f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:37:33.895292 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3e088-94c9-45f7-be76-94237682184f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:37:33.897007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3e088-94c9-45f7-be76-94237682184f service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:37:33.907439 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1152/2291] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:37:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:37:36.360170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=38 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:37:38.525191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff33da-eb50-44fd-af48-9e84f78cc8af service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:37:38.526796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ffff33da-eb50-44fd-af48-9e84f78cc8af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:37:38.542662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff33da-eb50-44fd-af48-9e84f78cc8af service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:37:38.549447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0df92-8d5c-4cbf-8ed5-12adca1c850a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:37:38.550137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0df92-8d5c-4cbf-8ed5-12adca1c850a service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:37:38.550423 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0df92-8d5c-4cbf-8ed5-12adca1c850a service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:37:38.550701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0df92-8d5c-4cbf-8ed5-12adca1c850a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:37:38.552575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-87d0df92-8d5c-4cbf-8ed5-12adca1c850a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:37:38.566227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d0df92-8d5c-4cbf-8ed5-12adca1c850a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:37:38.567871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d0df92-8d5c-4cbf-8ed5-12adca1c850a service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:37:38.578898 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1140/2292] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:37:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:37:43.903152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=28 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:37:48.574446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=6 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:39:08.330895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3e088-94c9-45f7-be76-94237682184f service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:08.332437 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d8c3e088-94c9-45f7-be76-94237682184f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:08.394721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3e088-94c9-45f7-be76-94237682184f service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:08.400681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc781523-79b4-4968-8f4c-7a8f97f2958b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:08.401478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc781523-79b4-4968-8f4c-7a8f97f2958b service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:08.401761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc781523-79b4-4968-8f4c-7a8f97f2958b service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:08.402033 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc781523-79b4-4968-8f4c-7a8f97f2958b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:08.403890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fc781523-79b4-4968-8f4c-7a8f97f2958b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:08.417186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc781523-79b4-4968-8f4c-7a8f97f2958b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:08.418838 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc781523-79b4-4968-8f4c-7a8f97f2958b service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:08.431251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1153/2293] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:39:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:39:18.425487 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=22 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:39:23.905827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0df92-8d5c-4cbf-8ed5-12adca1c850a service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:23.917149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0df92-8d5c-4cbf-8ed5-12adca1c850a service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:23.921523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404937c9-54e1-4ed7-a344-2ea682834689 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:23.922238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404937c9-54e1-4ed7-a344-2ea682834689 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:23.922599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404937c9-54e1-4ed7-a344-2ea682834689 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:23.922909 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404937c9-54e1-4ed7-a344-2ea682834689 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:23.925006 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-404937c9-54e1-4ed7-a344-2ea682834689 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:24.001866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc781523-79b4-4968-8f4c-7a8f97f2958b service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:24.011494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc781523-79b4-4968-8f4c-7a8f97f2958b service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:24.014856 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404937c9-54e1-4ed7-a344-2ea682834689 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:24.015679 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c99d40-3e5a-477a-8eff-589ba7b79fc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:24.016444 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404937c9-54e1-4ed7-a344-2ea682834689 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:24.016669 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c99d40-3e5a-477a-8eff-589ba7b79fc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:24.016890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c99d40-3e5a-477a-8eff-589ba7b79fc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:24.017171 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c99d40-3e5a-477a-8eff-589ba7b79fc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:24.019216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-38c99d40-3e5a-477a-8eff-589ba7b79fc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:24.028370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1141/2294] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:39:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:39:24.112633 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c99d40-3e5a-477a-8eff-589ba7b79fc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:24.114585 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c99d40-3e5a-477a-8eff-589ba7b79fc3 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:24.126028 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1154/2295] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:39:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:39:24.316706 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404937c9-54e1-4ed7-a344-2ea682834689 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:24.328252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404937c9-54e1-4ed7-a344-2ea682834689 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:24.332735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebb622-aebc-4730-8a11-ca9e441c1c70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:24.333465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebb622-aebc-4730-8a11-ca9e441c1c70 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:24.333746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebb622-aebc-4730-8a11-ca9e441c1c70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:24.333947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebb622-aebc-4730-8a11-ca9e441c1c70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:24.335779 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3cebb622-aebc-4730-8a11-ca9e441c1c70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:24.435776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cebb622-aebc-4730-8a11-ca9e441c1c70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:24.437351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cebb622-aebc-4730-8a11-ca9e441c1c70 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:24.458670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1142/2296] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:39:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:39:24.616610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c99d40-3e5a-477a-8eff-589ba7b79fc3 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:24.629361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c99d40-3e5a-477a-8eff-589ba7b79fc3 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:24.633137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2021eb4-f443-49cc-8de4-551eecebddc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:24.634025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2021eb4-f443-49cc-8de4-551eecebddc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:24.634307 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2021eb4-f443-49cc-8de4-551eecebddc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:24.634579 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2021eb4-f443-49cc-8de4-551eecebddc0 admin admin] PATH_INFO: `/v3/users/113b1059439a4cf98ba357053425830c` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:24.638034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2021eb4-f443-49cc-8de4-551eecebddc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=113b1059439a4cf98ba357053425830c)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:24.640304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2021eb4-f443-49cc-8de4-551eecebddc0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:24.710315 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f2021eb4-f443-49cc-8de4-551eecebddc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '113b1059439a4cf98ba357053425830c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:24.711581 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f2021eb4-f443-49cc-8de4-551eecebddc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '113b1059439a4cf98ba357053425830c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:24.725758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f2021eb4-f443-49cc-8de4-551eecebddc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '113b1059439a4cf98ba357053425830c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:24.742499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f2021eb4-f443-49cc-8de4-551eecebddc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '113b1059439a4cf98ba357053425830c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:24.751119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f2021eb4-f443-49cc-8de4-551eecebddc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '113b1059439a4cf98ba357053425830c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:24.758654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1155/2297] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:39:24 2022] DELETE /identity/v3/users/113b1059439a4cf98ba357053425830c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:39:24.763530 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce64a36-63ad-4047-843b-be344e1421df None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:24.763820 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce64a36-63ad-4047-843b-be344e1421df None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:24.764128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce64a36-63ad-4047-843b-be344e1421df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:24.783842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-dce64a36-63ad-4047-843b-be344e1421df None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:39:24.785198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dce64a36-63ad-4047-843b-be344e1421df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:24.846989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:39:24.860820 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1143/2298] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:39:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:39:24.873166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2021eb4-f443-49cc-8de4-551eecebddc0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:24.875169 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f2021eb4-f443-49cc-8de4-551eecebddc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:24.961297 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2021eb4-f443-49cc-8de4-551eecebddc0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:24.965288 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009b9c0-db4d-4917-93e0-c1781b3bc46c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:24.965943 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009b9c0-db4d-4917-93e0-c1781b3bc46c service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:24.966232 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009b9c0-db4d-4917-93e0-c1781b3bc46c service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:24.966518 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009b9c0-db4d-4917-93e0-c1781b3bc46c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:24.972933 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:39:24.978325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009b9c0-db4d-4917-93e0-c1781b3bc46c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:24.980347 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009b9c0-db4d-4917-93e0-c1781b3bc46c service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:25.000426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1156/2299] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:39:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:39:25.343961 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce64a36-63ad-4047-843b-be344e1421df None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:25.345394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dce64a36-63ad-4047-843b-be344e1421df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:25.402206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce64a36-63ad-4047-843b-be344e1421df None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:25.414140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2875b05-5378-428b-8b64-dc75d7c0e9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:25.415022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2875b05-5378-428b-8b64-dc75d7c0e9d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:25.415479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2875b05-5378-428b-8b64-dc75d7c0e9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:25.415791 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2875b05-5378-428b-8b64-dc75d7c0e9d3 admin admin] PATH_INFO: `/v3/projects/65b65d9f24b442d0aebce4ee10022176` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:25.417145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2875b05-5378-428b-8b64-dc75d7c0e9d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b65d9f24b442d0aebce4ee10022176)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:25.419168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2875b05-5378-428b-8b64-dc75d7c0e9d3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:25.478281 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b2875b05-5378-428b-8b64-dc75d7c0e9d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b65d9f24b442d0aebce4ee10022176', 'request_id': 'req-b2875b05-5378-428b-8b64-dc75d7c0e9d3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:25.485754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b2875b05-5378-428b-8b64-dc75d7c0e9d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b65d9f24b442d0aebce4ee10022176', 'request_id': 'req-b2875b05-5378-428b-8b64-dc75d7c0e9d3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:25.495037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b2875b05-5378-428b-8b64-dc75d7c0e9d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b65d9f24b442d0aebce4ee10022176', 'request_id': 'req-b2875b05-5378-428b-8b64-dc75d7c0e9d3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:25.506935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b2875b05-5378-428b-8b64-dc75d7c0e9d3 admin admin] The token cache is being invalidate because project 65b65d9f24b442d0aebce4ee10022176 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:39:25.507293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b2875b05-5378-428b-8b64-dc75d7c0e9d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:25.507883 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b2875b05-5378-428b-8b64-dc75d7c0e9d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:25.509494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1144/2300] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:39:25 2022] DELETE /identity/v3/projects/65b65d9f24b442d0aebce4ee10022176 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:39:25.513896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009b9c0-db4d-4917-93e0-c1781b3bc46c service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:25.539132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009b9c0-db4d-4917-93e0-c1781b3bc46c service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:25.656918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05817713-dd44-4990-8e56-a2dc154392f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:25.658121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05817713-dd44-4990-8e56-a2dc154392f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:25.658480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05817713-dd44-4990-8e56-a2dc154392f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:25.658831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05817713-dd44-4990-8e56-a2dc154392f9 admin admin] PATH_INFO: `/v3/users/9d610a4ba2764e6692530c6fca30e4e9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:25.660526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05817713-dd44-4990-8e56-a2dc154392f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d610a4ba2764e6692530c6fca30e4e9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:25.663939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05817713-dd44-4990-8e56-a2dc154392f9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:25.736830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-05817713-dd44-4990-8e56-a2dc154392f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d610a4ba2764e6692530c6fca30e4e9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:25.737495 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-05817713-dd44-4990-8e56-a2dc154392f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d610a4ba2764e6692530c6fca30e4e9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:25.749012 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-05817713-dd44-4990-8e56-a2dc154392f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d610a4ba2764e6692530c6fca30e4e9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:25.763499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-05817713-dd44-4990-8e56-a2dc154392f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d610a4ba2764e6692530c6fca30e4e9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:25.771854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-05817713-dd44-4990-8e56-a2dc154392f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d610a4ba2764e6692530c6fca30e4e9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:25.778231 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1157/2301] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:39:25 2022] DELETE /identity/v3/users/9d610a4ba2764e6692530c6fca30e4e9 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:39:26.355892 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2875b05-5378-428b-8b64-dc75d7c0e9d3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:26.357403 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b2875b05-5378-428b-8b64-dc75d7c0e9d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:26.383134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2875b05-5378-428b-8b64-dc75d7c0e9d3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:26.464075 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d115a-e793-4dd9-a0b9-9124c4a9c162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:26.465048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d115a-e793-4dd9-a0b9-9124c4a9c162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:26.465405 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d115a-e793-4dd9-a0b9-9124c4a9c162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:26.465751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d115a-e793-4dd9-a0b9-9124c4a9c162 admin admin] PATH_INFO: `/v3/projects/731597dafaf44528a7c44343e2a961fd` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:26.467057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d115a-e793-4dd9-a0b9-9124c4a9c162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=731597dafaf44528a7c44343e2a961fd)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:26.469121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d115a-e793-4dd9-a0b9-9124c4a9c162 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:26.524458 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a55d115a-e793-4dd9-a0b9-9124c4a9c162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '731597dafaf44528a7c44343e2a961fd', 'request_id': 'req-a55d115a-e793-4dd9-a0b9-9124c4a9c162'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:26.531865 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a55d115a-e793-4dd9-a0b9-9124c4a9c162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '731597dafaf44528a7c44343e2a961fd', 'request_id': 'req-a55d115a-e793-4dd9-a0b9-9124c4a9c162'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:26.534738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:39:26.543348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a55d115a-e793-4dd9-a0b9-9124c4a9c162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '731597dafaf44528a7c44343e2a961fd', 'request_id': 'req-a55d115a-e793-4dd9-a0b9-9124c4a9c162'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:26.557202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a55d115a-e793-4dd9-a0b9-9124c4a9c162 admin admin] The token cache is being invalidate because project 731597dafaf44528a7c44343e2a961fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:39:26.557488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a55d115a-e793-4dd9-a0b9-9124c4a9c162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:26.558148 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a55d115a-e793-4dd9-a0b9-9124c4a9c162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:26.559705 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1145/2302] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:39:26 2022] DELETE /identity/v3/projects/731597dafaf44528a7c44343e2a961fd => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:39:26.564966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38f43f-d2cc-4bef-8cf7-5e328bb34e71 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:26.565490 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38f43f-d2cc-4bef-8cf7-5e328bb34e71 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:26.565664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38f43f-d2cc-4bef-8cf7-5e328bb34e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:26.585899 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-2b38f43f-d2cc-4bef-8cf7-5e328bb34e71 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:39:26.587647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2b38f43f-d2cc-4bef-8cf7-5e328bb34e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:26.654196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:39:26.672290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1158/2303] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:39:26.674679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9a595-fa6d-4542-8b52-c1d8641456f5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:26.675456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9a595-fa6d-4542-8b52-c1d8641456f5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:26.675748 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9a595-fa6d-4542-8b52-c1d8641456f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:26.696308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a4a9a595-fa6d-4542-8b52-c1d8641456f5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:39:26.697554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a4a9a595-fa6d-4542-8b52-c1d8641456f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:26.705528 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1146/2304] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:39:26.710629 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38f43f-d2cc-4bef-8cf7-5e328bb34e71 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:26.721129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38f43f-d2cc-4bef-8cf7-5e328bb34e71 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:26.726417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02619806-96d6-4b63-b984-9145e14380cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:26.727134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02619806-96d6-4b63-b984-9145e14380cf admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:26.727402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02619806-96d6-4b63-b984-9145e14380cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:26.727690 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02619806-96d6-4b63-b984-9145e14380cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:26.728540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02619806-96d6-4b63-b984-9145e14380cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:39:26.728802 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02619806-96d6-4b63-b984-9145e14380cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:26.730399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02619806-96d6-4b63-b984-9145e14380cf admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:26.746247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1159/2305] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:39:26.747763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9a595-fa6d-4542-8b52-c1d8641456f5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:26.758191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9a595-fa6d-4542-8b52-c1d8641456f5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:26.762969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edcb72-5c40-4b5a-adca-6bc4fc386943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:26.763631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edcb72-5c40-4b5a-adca-6bc4fc386943 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:26.763946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edcb72-5c40-4b5a-adca-6bc4fc386943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:26.764220 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edcb72-5c40-4b5a-adca-6bc4fc386943 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:26.764919 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44edcb72-5c40-4b5a-adca-6bc4fc386943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:26.766866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44edcb72-5c40-4b5a-adca-6bc4fc386943 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:26.802838 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1147/2306] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:39:26 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:39:26.807144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02619806-96d6-4b63-b984-9145e14380cf admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:26.820602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02619806-96d6-4b63-b984-9145e14380cf admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:26.897489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563ee0e-2f2d-4721-baba-4cee0a2bc611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:26.898520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563ee0e-2f2d-4721-baba-4cee0a2bc611 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:26.898809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563ee0e-2f2d-4721-baba-4cee0a2bc611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:26.899092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563ee0e-2f2d-4721-baba-4cee0a2bc611 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:26.899810 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563ee0e-2f2d-4721-baba-4cee0a2bc611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:26.902408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563ee0e-2f2d-4721-baba-4cee0a2bc611 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:26.925326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1160/2307] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:39:26 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:39:26.929891 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edcb72-5c40-4b5a-adca-6bc4fc386943 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:26.940437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edcb72-5c40-4b5a-adca-6bc4fc386943 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:26.946930 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1530f-0e8e-4d7b-a421-81ffaefb9bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:26.947602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1530f-0e8e-4d7b-a421-81ffaefb9bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:26.948347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1530f-0e8e-4d7b-a421-81ffaefb9bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:26.948347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1530f-0e8e-4d7b-a421-81ffaefb9bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:26.948934 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1530f-0e8e-4d7b-a421-81ffaefb9bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:26.950624 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1530f-0e8e-4d7b-a421-81ffaefb9bc1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:26.964684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1148/2308] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:39:26.968842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563ee0e-2f2d-4721-baba-4cee0a2bc611 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:26.980474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563ee0e-2f2d-4721-baba-4cee0a2bc611 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:26.987079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91df9e4-0cf9-4c42-a7af-b5dc66cd2642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:26.987767 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91df9e4-0cf9-4c42-a7af-b5dc66cd2642 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:26.988458 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91df9e4-0cf9-4c42-a7af-b5dc66cd2642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:26.988458 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91df9e4-0cf9-4c42-a7af-b5dc66cd2642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:26.990079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91df9e4-0cf9-4c42-a7af-b5dc66cd2642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:26.991802 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91df9e4-0cf9-4c42-a7af-b5dc66cd2642 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:27.003728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1161/2309] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:39:27.008504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1530f-0e8e-4d7b-a421-81ffaefb9bc1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:27.019147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1530f-0e8e-4d7b-a421-81ffaefb9bc1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:27.024685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84aec8-5281-4f24-909a-12b93e28bf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:27.025953 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84aec8-5281-4f24-909a-12b93e28bf9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:27.026264 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84aec8-5281-4f24-909a-12b93e28bf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:27.026517 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84aec8-5281-4f24-909a-12b93e28bf9e admin admin] PATH_INFO: `/v3/projects/3b63984d8ef44f72b723e622f36132bd/users/8ff9e7eef23c46b4abeb05da90a0cf94/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:27.046247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84aec8-5281-4f24-909a-12b93e28bf9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b63984d8ef44f72b723e622f36132bd, user_id=8ff9e7eef23c46b4abeb05da90a0cf94, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:27.049310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84aec8-5281-4f24-909a-12b93e28bf9e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:27.068815 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1149/2310] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:39:27 2022] PUT /identity/v3/projects/3b63984d8ef44f72b723e622f36132bd/users/8ff9e7eef23c46b4abeb05da90a0cf94/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:39:27.080438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04afe50-2352-4228-a74f-43cec84f3b89 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:27.080840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04afe50-2352-4228-a74f-43cec84f3b89 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:27.081213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04afe50-2352-4228-a74f-43cec84f3b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:27.114962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a04afe50-2352-4228-a74f-43cec84f3b89 None None] MFA Rules not processed for user `8ff9e7eef23c46b4abeb05da90a0cf94`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:39:27.116365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a04afe50-2352-4228-a74f-43cec84f3b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:27.203992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1162/2311] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:39:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:39:28.571136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84aec8-5281-4f24-909a-12b93e28bf9e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:28.572997 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fe84aec8-5281-4f24-909a-12b93e28bf9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:28.648159 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84aec8-5281-4f24-909a-12b93e28bf9e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:28.652961 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf33e8-8b7a-4dce-80ee-016f2c731e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:28.653859 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf33e8-8b7a-4dce-80ee-016f2c731e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:28.654152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf33e8-8b7a-4dce-80ee-016f2c731e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:28.654432 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf33e8-8b7a-4dce-80ee-016f2c731e97 admin admin] PATH_INFO: `/v3/users/e717fb256de24adb8ee6815338a94e5d` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:28.655718 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf33e8-8b7a-4dce-80ee-016f2c731e97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e717fb256de24adb8ee6815338a94e5d)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:28.657761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf33e8-8b7a-4dce-80ee-016f2c731e97 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:28.710445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3ebf33e8-8b7a-4dce-80ee-016f2c731e97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e717fb256de24adb8ee6815338a94e5d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:28.725911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3ebf33e8-8b7a-4dce-80ee-016f2c731e97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e717fb256de24adb8ee6815338a94e5d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:28.733749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3ebf33e8-8b7a-4dce-80ee-016f2c731e97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e717fb256de24adb8ee6815338a94e5d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:28.746814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3ebf33e8-8b7a-4dce-80ee-016f2c731e97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e717fb256de24adb8ee6815338a94e5d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:28.747568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3ebf33e8-8b7a-4dce-80ee-016f2c731e97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e717fb256de24adb8ee6815338a94e5d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:28.749893 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:39:28.755257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1150/2312] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:39:28 2022] DELETE /identity/v3/users/e717fb256de24adb8ee6815338a94e5d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:39:28.939899 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04afe50-2352-4228-a74f-43cec84f3b89 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:28.953440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04afe50-2352-4228-a74f-43cec84f3b89 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:29.008020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:39:29.016699 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ca7e2-ceae-40f2-a65c-10048c79b872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:29.017734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ca7e2-ceae-40f2-a65c-10048c79b872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:29.017882 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ca7e2-ceae-40f2-a65c-10048c79b872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:29.018159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ca7e2-ceae-40f2-a65c-10048c79b872 admin admin] PATH_INFO: `/v3/projects/11c9eb819852493baadbcc11ecce2914` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:29.019479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ca7e2-ceae-40f2-a65c-10048c79b872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c9eb819852493baadbcc11ecce2914)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:29.021886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ca7e2-ceae-40f2-a65c-10048c79b872 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:29.079199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f81ca7e2-ceae-40f2-a65c-10048c79b872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c9eb819852493baadbcc11ecce2914', 'request_id': 'req-f81ca7e2-ceae-40f2-a65c-10048c79b872'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:29.086027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f81ca7e2-ceae-40f2-a65c-10048c79b872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c9eb819852493baadbcc11ecce2914', 'request_id': 'req-f81ca7e2-ceae-40f2-a65c-10048c79b872'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:29.093119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f81ca7e2-ceae-40f2-a65c-10048c79b872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c9eb819852493baadbcc11ecce2914', 'request_id': 'req-f81ca7e2-ceae-40f2-a65c-10048c79b872'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:29.107814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f81ca7e2-ceae-40f2-a65c-10048c79b872 admin admin] The token cache is being invalidate because project 11c9eb819852493baadbcc11ecce2914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:39:29.108141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f81ca7e2-ceae-40f2-a65c-10048c79b872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:29.108825 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f81ca7e2-ceae-40f2-a65c-10048c79b872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:39:29.110413 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1163/2313] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:39:28 2022] DELETE /identity/v3/projects/11c9eb819852493baadbcc11ecce2914 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:39:29.117086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591e939-aa8f-4cd5-a5e0-b11393306ef5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:29.117428 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591e939-aa8f-4cd5-a5e0-b11393306ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:29.117717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591e939-aa8f-4cd5-a5e0-b11393306ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:29.138721 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b591e939-aa8f-4cd5-a5e0-b11393306ef5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:39:29.140096 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b591e939-aa8f-4cd5-a5e0-b11393306ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:29.201747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1151/2314] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:39:29.210554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed08e6f-d91c-4b60-9724-9f3cc7ea105d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:29.210831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed08e6f-d91c-4b60-9724-9f3cc7ea105d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:29.211103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed08e6f-d91c-4b60-9724-9f3cc7ea105d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:29.230011 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-9ed08e6f-d91c-4b60-9724-9f3cc7ea105d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:39:29.231236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9ed08e6f-d91c-4b60-9724-9f3cc7ea105d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:29.239670 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1164/2315] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:39:29.244006 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591e939-aa8f-4cd5-a5e0-b11393306ef5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:29.253859 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591e939-aa8f-4cd5-a5e0-b11393306ef5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:29.259291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7222771-a906-4003-9675-1d7d46146f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:29.260036 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7222771-a906-4003-9675-1d7d46146f6e admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:29.260324 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7222771-a906-4003-9675-1d7d46146f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:29.260602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7222771-a906-4003-9675-1d7d46146f6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:29.261336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7222771-a906-4003-9675-1d7d46146f6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:39:29.261647 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7222771-a906-4003-9675-1d7d46146f6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:29.263271 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7222771-a906-4003-9675-1d7d46146f6e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:29.275709 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1152/2316] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:39:29.280122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed08e6f-d91c-4b60-9724-9f3cc7ea105d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:29.290098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed08e6f-d91c-4b60-9724-9f3cc7ea105d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:29.294787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01762c4b-affd-4a52-9be8-cfac6e0b4813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:29.295443 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01762c4b-affd-4a52-9be8-cfac6e0b4813 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:29.295729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01762c4b-affd-4a52-9be8-cfac6e0b4813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:29.296298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01762c4b-affd-4a52-9be8-cfac6e0b4813 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:29.297002 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01762c4b-affd-4a52-9be8-cfac6e0b4813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:29.298945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01762c4b-affd-4a52-9be8-cfac6e0b4813 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:29.326524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1165/2317] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:39:29 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:39:29.331037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7222771-a906-4003-9675-1d7d46146f6e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:29.341554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7222771-a906-4003-9675-1d7d46146f6e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:29.402601 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072d26a-e17f-4139-8c29-9a2a0c5b86b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:29.403274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072d26a-e17f-4139-8c29-9a2a0c5b86b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:29.403559 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072d26a-e17f-4139-8c29-9a2a0c5b86b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:29.403835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072d26a-e17f-4139-8c29-9a2a0c5b86b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:29.404577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072d26a-e17f-4139-8c29-9a2a0c5b86b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:29.406528 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072d26a-e17f-4139-8c29-9a2a0c5b86b8 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:29.432566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1153/2318] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:39:29 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:39:29.436801 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01762c4b-affd-4a52-9be8-cfac6e0b4813 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:29.448400 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01762c4b-affd-4a52-9be8-cfac6e0b4813 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:29.453208 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d69f9a-2c9e-418a-b2f4-498ace826d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:29.453818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d69f9a-2c9e-418a-b2f4-498ace826d7b admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:29.454099 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d69f9a-2c9e-418a-b2f4-498ace826d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:29.454373 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d69f9a-2c9e-418a-b2f4-498ace826d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:29.455108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d69f9a-2c9e-418a-b2f4-498ace826d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:29.456787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d69f9a-2c9e-418a-b2f4-498ace826d7b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:29.468192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1166/2319] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:39:29.472492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072d26a-e17f-4139-8c29-9a2a0c5b86b8 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:29.483777 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072d26a-e17f-4139-8c29-9a2a0c5b86b8 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:29.489750 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a905f0f-0e90-427a-8e34-63b5ab14a449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:29.490447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a905f0f-0e90-427a-8e34-63b5ab14a449 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:29.490735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a905f0f-0e90-427a-8e34-63b5ab14a449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:29.491017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a905f0f-0e90-427a-8e34-63b5ab14a449 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:29.491768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a905f0f-0e90-427a-8e34-63b5ab14a449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:29.493514 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a905f0f-0e90-427a-8e34-63b5ab14a449 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:29.505107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1154/2320] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:39:29.509910 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d69f9a-2c9e-418a-b2f4-498ace826d7b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:29.520623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d69f9a-2c9e-418a-b2f4-498ace826d7b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:29.525468 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6cce09-c09b-47cd-b6fc-699b24d07ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:29.526374 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6cce09-c09b-47cd-b6fc-699b24d07ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:29.526651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6cce09-c09b-47cd-b6fc-699b24d07ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:29.526919 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6cce09-c09b-47cd-b6fc-699b24d07ee6 admin admin] PATH_INFO: `/v3/projects/e1cff23fc1d24489af261e024b5bd19f/users/d6c1fda6b45a4df088bbd0b7d41fe31e/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:29.547160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6cce09-c09b-47cd-b6fc-699b24d07ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1cff23fc1d24489af261e024b5bd19f, user_id=d6c1fda6b45a4df088bbd0b7d41fe31e, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:29.552718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6cce09-c09b-47cd-b6fc-699b24d07ee6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:29.569791 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1167/2321] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:39:29 2022] PUT /identity/v3/projects/e1cff23fc1d24489af261e024b5bd19f/users/d6c1fda6b45a4df088bbd0b7d41fe31e/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:39:29.580178 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab82c4b-0687-4de8-9db9-775cdd86603a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:29.580732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab82c4b-0687-4de8-9db9-775cdd86603a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:29.580732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab82c4b-0687-4de8-9db9-775cdd86603a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:29.608211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-7ab82c4b-0687-4de8-9db9-775cdd86603a None None] MFA Rules not processed for user `d6c1fda6b45a4df088bbd0b7d41fe31e`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:39:29.609445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7ab82c4b-0687-4de8-9db9-775cdd86603a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:29.673689 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:39:29.687775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1155/2322] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:39:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:39:29.699857 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6cce09-c09b-47cd-b6fc-699b24d07ee6 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:39:29.702166 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7b6cce09-c09b-47cd-b6fc-699b24d07ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:39:29.782959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6cce09-c09b-47cd-b6fc-699b24d07ee6 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:39:29.789361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee012ac-416d-4bbd-8cce-22336780e95c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:39:29.790015 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee012ac-416d-4bbd-8cce-22336780e95c service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:39:29.790300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee012ac-416d-4bbd-8cce-22336780e95c service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:39:29.790577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee012ac-416d-4bbd-8cce-22336780e95c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:39:29.799181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee012ac-416d-4bbd-8cce-22336780e95c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:39:29.800886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee012ac-416d-4bbd-8cce-22336780e95c service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:39:29.811071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1168/2323] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:39:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:39:39.685304 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:39:39.807035 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:24.333464 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab82c4b-0687-4de8-9db9-775cdd86603a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:24.334846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7ab82c4b-0687-4de8-9db9-775cdd86603a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:24.415622 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab82c4b-0687-4de8-9db9-775cdd86603a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:24.419410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfbc99-5ef5-4538-8c66-2a580bb25972 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:24.420087 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfbc99-5ef5-4538-8c66-2a580bb25972 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:24.420446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfbc99-5ef5-4538-8c66-2a580bb25972 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:24.421019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfbc99-5ef5-4538-8c66-2a580bb25972 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:24.422792 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d9cfbc99-5ef5-4538-8c66-2a580bb25972 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:24.499519 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfbc99-5ef5-4538-8c66-2a580bb25972 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:24.501008 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfbc99-5ef5-4538-8c66-2a580bb25972 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:24.510157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1156/2324] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:41:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:41:24.591938 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee012ac-416d-4bbd-8cce-22336780e95c service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:24.612091 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee012ac-416d-4bbd-8cce-22336780e95c service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:24.615756 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd4549-cab8-4ce7-8da6-dbbdd863dd03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:24.616422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd4549-cab8-4ce7-8da6-dbbdd863dd03 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:24.616705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd4549-cab8-4ce7-8da6-dbbdd863dd03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:24.616981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd4549-cab8-4ce7-8da6-dbbdd863dd03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:24.620105 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-74dd4549-cab8-4ce7-8da6-dbbdd863dd03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:24.683105 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd4549-cab8-4ce7-8da6-dbbdd863dd03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:24.684820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd4549-cab8-4ce7-8da6-dbbdd863dd03 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:24.695388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1169/2325] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:41:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:41:24.742353 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfbc99-5ef5-4538-8c66-2a580bb25972 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:24.752615 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfbc99-5ef5-4538-8c66-2a580bb25972 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:24.756261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fd5c6-c78e-47d5-a53e-a6cc56d19401 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:24.756898 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fd5c6-c78e-47d5-a53e-a6cc56d19401 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:24.757179 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fd5c6-c78e-47d5-a53e-a6cc56d19401 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:24.757530 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fd5c6-c78e-47d5-a53e-a6cc56d19401 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:24.760123 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-171fd5c6-c78e-47d5-a53e-a6cc56d19401 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:24.825919 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171fd5c6-c78e-47d5-a53e-a6cc56d19401 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:24.827312 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171fd5c6-c78e-47d5-a53e-a6cc56d19401 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:24.836614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1157/2326] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:41:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:41:24.852965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd4549-cab8-4ce7-8da6-dbbdd863dd03 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:24.861459 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd4549-cab8-4ce7-8da6-dbbdd863dd03 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:24.865055 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111f52f-a208-49a4-a6cd-48a631128e95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:24.865716 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111f52f-a208-49a4-a6cd-48a631128e95 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:24.865997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111f52f-a208-49a4-a6cd-48a631128e95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:24.866273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111f52f-a208-49a4-a6cd-48a631128e95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:24.868544 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7111f52f-a208-49a4-a6cd-48a631128e95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:24.889181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111f52f-a208-49a4-a6cd-48a631128e95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:24.890599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111f52f-a208-49a4-a6cd-48a631128e95 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:24.911153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1170/2327] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:41:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:41:25.054801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fd5c6-c78e-47d5-a53e-a6cc56d19401 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:25.059477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111f52f-a208-49a4-a6cd-48a631128e95 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:25.061182 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7111f52f-a208-49a4-a6cd-48a631128e95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:25.063730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fd5c6-c78e-47d5-a53e-a6cc56d19401 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:25.067730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39306d-1243-4a2c-9db1-75d43cfe97ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:25.068403 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39306d-1243-4a2c-9db1-75d43cfe97ba service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:25.068684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39306d-1243-4a2c-9db1-75d43cfe97ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:25.068958 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39306d-1243-4a2c-9db1-75d43cfe97ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:25.070696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9b39306d-1243-4a2c-9db1-75d43cfe97ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:25.085268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39306d-1243-4a2c-9db1-75d43cfe97ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:25.087375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39306d-1243-4a2c-9db1-75d43cfe97ba service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:25.098798 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1158/2328] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:41:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:41:25.135879 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111f52f-a208-49a4-a6cd-48a631128e95 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:25.143272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248caed8-842b-4670-9ab9-04cb913699e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:25.144190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248caed8-842b-4670-9ab9-04cb913699e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:25.144475 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248caed8-842b-4670-9ab9-04cb913699e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:25.144753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248caed8-842b-4670-9ab9-04cb913699e7 admin admin] PATH_INFO: `/v3/users/8ff9e7eef23c46b4abeb05da90a0cf94` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:25.145961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248caed8-842b-4670-9ab9-04cb913699e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ff9e7eef23c46b4abeb05da90a0cf94)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:25.148060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248caed8-842b-4670-9ab9-04cb913699e7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:25.205214 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-248caed8-842b-4670-9ab9-04cb913699e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ff9e7eef23c46b4abeb05da90a0cf94'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:25.205876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-248caed8-842b-4670-9ab9-04cb913699e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ff9e7eef23c46b4abeb05da90a0cf94'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:25.216153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-248caed8-842b-4670-9ab9-04cb913699e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ff9e7eef23c46b4abeb05da90a0cf94'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:25.223586 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:25.233081 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-248caed8-842b-4670-9ab9-04cb913699e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ff9e7eef23c46b4abeb05da90a0cf94'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:25.241045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-248caed8-842b-4670-9ab9-04cb913699e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ff9e7eef23c46b4abeb05da90a0cf94'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:25.253259 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39306d-1243-4a2c-9db1-75d43cfe97ba service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:25.253554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1171/2329] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:41:25 2022] DELETE /identity/v3/users/8ff9e7eef23c46b4abeb05da90a0cf94 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:41:25.257751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aa12a-5d08-45a3-bb2e-1b57bb9812c1 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:25.258046 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aa12a-5d08-45a3-bb2e-1b57bb9812c1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:25.259057 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aa12a-5d08-45a3-bb2e-1b57bb9812c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:25.265160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39306d-1243-4a2c-9db1-75d43cfe97ba service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:25.270074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:25.276711 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c54aa12a-5d08-45a3-bb2e-1b57bb9812c1 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:25.277972 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c54aa12a-5d08-45a3-bb2e-1b57bb9812c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:25.327175 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dd716-7d0d-4e0b-997b-6a19063b286b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:25.328205 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dd716-7d0d-4e0b-997b-6a19063b286b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:25.328495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dd716-7d0d-4e0b-997b-6a19063b286b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:25.328796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dd716-7d0d-4e0b-997b-6a19063b286b admin admin] PATH_INFO: `/v3/users/d6c1fda6b45a4df088bbd0b7d41fe31e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:25.330003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550dd716-7d0d-4e0b-997b-6a19063b286b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6c1fda6b45a4df088bbd0b7d41fe31e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:25.331984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550dd716-7d0d-4e0b-997b-6a19063b286b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:25.338886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1172/2330] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:41:25.348469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54aa12a-5d08-45a3-bb2e-1b57bb9812c1 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:25.349906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c54aa12a-5d08-45a3-bb2e-1b57bb9812c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:25.382911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-550dd716-7d0d-4e0b-997b-6a19063b286b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6c1fda6b45a4df088bbd0b7d41fe31e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:25.392157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-550dd716-7d0d-4e0b-997b-6a19063b286b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6c1fda6b45a4df088bbd0b7d41fe31e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:25.399675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-550dd716-7d0d-4e0b-997b-6a19063b286b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6c1fda6b45a4df088bbd0b7d41fe31e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:25.411857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-550dd716-7d0d-4e0b-997b-6a19063b286b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6c1fda6b45a4df088bbd0b7d41fe31e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:25.412570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-550dd716-7d0d-4e0b-997b-6a19063b286b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6c1fda6b45a4df088bbd0b7d41fe31e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:25.418979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1159/2331] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:41:25 2022] DELETE /identity/v3/users/d6c1fda6b45a4df088bbd0b7d41fe31e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:41:25.424481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9aba7-8d67-4805-ba43-5ad78bc07e14 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:25.424897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9aba7-8d67-4805-ba43-5ad78bc07e14 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:25.425179 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9aba7-8d67-4805-ba43-5ad78bc07e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:25.429016 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54aa12a-5d08-45a3-bb2e-1b57bb9812c1 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:25.445592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c0b9aba7-8d67-4805-ba43-5ad78bc07e14 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:25.447549 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c0b9aba7-8d67-4805-ba43-5ad78bc07e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:25.486651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ee1ed-ba26-4d4d-a38c-b7095d1d5460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:25.487305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ee1ed-ba26-4d4d-a38c-b7095d1d5460 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:25.487592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ee1ed-ba26-4d4d-a38c-b7095d1d5460 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:25.488254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ee1ed-ba26-4d4d-a38c-b7095d1d5460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:25.490165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-979ee1ed-ba26-4d4d-a38c-b7095d1d5460 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:25.511119 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1160/2332] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:41:25.520873 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9aba7-8d67-4805-ba43-5ad78bc07e14 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:25.522312 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c0b9aba7-8d67-4805-ba43-5ad78bc07e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:25.534142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9aba7-8d67-4805-ba43-5ad78bc07e14 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:25.535415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:25.538965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211cfd-1eac-4ba7-ab3a-9804766dc2e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:25.539621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211cfd-1eac-4ba7-ab3a-9804766dc2e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:25.540162 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211cfd-1eac-4ba7-ab3a-9804766dc2e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:25.540488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211cfd-1eac-4ba7-ab3a-9804766dc2e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:25.552277 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df211cfd-1eac-4ba7-ab3a-9804766dc2e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:25.553768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df211cfd-1eac-4ba7-ab3a-9804766dc2e9 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:25.560447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ee1ed-ba26-4d4d-a38c-b7095d1d5460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:25.562154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ee1ed-ba26-4d4d-a38c-b7095d1d5460 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:25.564341 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1161/2333] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:41:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:41:25.572288 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1173/2334] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:41:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:41:26.044341 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211cfd-1eac-4ba7-ab3a-9804766dc2e9 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.046252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-df211cfd-1eac-4ba7-ab3a-9804766dc2e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:26.052848 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ee1ed-ba26-4d4d-a38c-b7095d1d5460 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.054366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-979ee1ed-ba26-4d4d-a38c-b7095d1d5460 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:26.065567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211cfd-1eac-4ba7-ab3a-9804766dc2e9 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:26.072194 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647bbafa-ea8a-43a8-a4cb-9ac9e24ee7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:26.073085 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ee1ed-ba26-4d4d-a38c-b7095d1d5460 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:26.077916 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec854f7-ea59-4148-a78d-da7ce6b65292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:26.078919 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec854f7-ea59-4148-a78d-da7ce6b65292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.088161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec854f7-ea59-4148-a78d-da7ce6b65292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.088647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec854f7-ea59-4148-a78d-da7ce6b65292 admin admin] PATH_INFO: `/v3/projects/e1cff23fc1d24489af261e024b5bd19f` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.090053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec854f7-ea59-4148-a78d-da7ce6b65292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1cff23fc1d24489af261e024b5bd19f)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:26.092045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec854f7-ea59-4148-a78d-da7ce6b65292 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:26.104878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bbafa-ea8a-43a8-a4cb-9ac9e24ee7f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.108102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bbafa-ea8a-43a8-a4cb-9ac9e24ee7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.110097 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647bbafa-ea8a-43a8-a4cb-9ac9e24ee7f9 admin admin] PATH_INFO: `/v3/projects/3b63984d8ef44f72b723e622f36132bd` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.120576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647bbafa-ea8a-43a8-a4cb-9ac9e24ee7f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b63984d8ef44f72b723e622f36132bd)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:26.122577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647bbafa-ea8a-43a8-a4cb-9ac9e24ee7f9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:26.144856 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:26.163265 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0ec854f7-ea59-4148-a78d-da7ce6b65292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1cff23fc1d24489af261e024b5bd19f', 'request_id': 'req-0ec854f7-ea59-4148-a78d-da7ce6b65292'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:26.171835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0ec854f7-ea59-4148-a78d-da7ce6b65292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1cff23fc1d24489af261e024b5bd19f', 'request_id': 'req-0ec854f7-ea59-4148-a78d-da7ce6b65292'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:26.178282 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0ec854f7-ea59-4148-a78d-da7ce6b65292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1cff23fc1d24489af261e024b5bd19f', 'request_id': 'req-0ec854f7-ea59-4148-a78d-da7ce6b65292'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:26.190157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-647bbafa-ea8a-43a8-a4cb-9ac9e24ee7f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b63984d8ef44f72b723e622f36132bd', 'request_id': 'req-647bbafa-ea8a-43a8-a4cb-9ac9e24ee7f9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:26.191740 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0ec854f7-ea59-4148-a78d-da7ce6b65292 admin admin] The token cache is being invalidate because project e1cff23fc1d24489af261e024b5bd19f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:41:26.192116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0ec854f7-ea59-4148-a78d-da7ce6b65292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:26.192747 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0ec854f7-ea59-4148-a78d-da7ce6b65292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:26.194323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1174/2335] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:41:26 2022] DELETE /identity/v3/projects/e1cff23fc1d24489af261e024b5bd19f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:41:26.199737 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dcc39-86c2-450a-bc91-e67e81cb8ca1 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.200060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dcc39-86c2-450a-bc91-e67e81cb8ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.200341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9dcc39-86c2-450a-bc91-e67e81cb8ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.200869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-647bbafa-ea8a-43a8-a4cb-9ac9e24ee7f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b63984d8ef44f72b723e622f36132bd', 'request_id': 'req-647bbafa-ea8a-43a8-a4cb-9ac9e24ee7f9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:26.209598 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-647bbafa-ea8a-43a8-a4cb-9ac9e24ee7f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b63984d8ef44f72b723e622f36132bd', 'request_id': 'req-647bbafa-ea8a-43a8-a4cb-9ac9e24ee7f9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:26.219775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6c9dcc39-86c2-450a-bc91-e67e81cb8ca1 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:26.221039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6c9dcc39-86c2-450a-bc91-e67e81cb8ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:26.221539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-647bbafa-ea8a-43a8-a4cb-9ac9e24ee7f9 admin admin] The token cache is being invalidate because project 3b63984d8ef44f72b723e622f36132bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:41:26.221906 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-647bbafa-ea8a-43a8-a4cb-9ac9e24ee7f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:26.222605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-647bbafa-ea8a-43a8-a4cb-9ac9e24ee7f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:26.224345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1162/2336] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:41:26 2022] DELETE /identity/v3/projects/3b63984d8ef44f72b723e622f36132bd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:41:26.229415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a665a-3860-4a5c-b8b6-18a7d96c52f2 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.229706 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a665a-3860-4a5c-b8b6-18a7d96c52f2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.229989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a665a-3860-4a5c-b8b6-18a7d96c52f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.249341 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-734a665a-3860-4a5c-b8b6-18a7d96c52f2 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:26.250560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-734a665a-3860-4a5c-b8b6-18a7d96c52f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:26.286955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1175/2337] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:41:26.295985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19096149-16a8-47b7-bb27-976e35e3eab9 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.295985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19096149-16a8-47b7-bb27-976e35e3eab9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.296297 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19096149-16a8-47b7-bb27-976e35e3eab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.316278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-19096149-16a8-47b7-bb27-976e35e3eab9 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:26.317514 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-19096149-16a8-47b7-bb27-976e35e3eab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:26.326450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1176/2338] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:41:26.326896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1163/2339] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:41:26.330781 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19096149-16a8-47b7-bb27-976e35e3eab9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.334795 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8c408-4e3a-4fd0-9649-70345f840d38 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.335045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8c408-4e3a-4fd0-9649-70345f840d38 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.335325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8c408-4e3a-4fd0-9649-70345f840d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.341739 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19096149-16a8-47b7-bb27-976e35e3eab9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:26.346374 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea15720-ac75-496a-aeea-d60ef3572131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:26.347036 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea15720-ac75-496a-aeea-d60ef3572131 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.347452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea15720-ac75-496a-aeea-d60ef3572131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.347737 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea15720-ac75-496a-aeea-d60ef3572131 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.348642 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea15720-ac75-496a-aeea-d60ef3572131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:41:26.348807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea15720-ac75-496a-aeea-d60ef3572131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:26.350442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea15720-ac75-496a-aeea-d60ef3572131 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:26.355434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-aaf8c408-4e3a-4fd0-9649-70345f840d38 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:26.356803 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-aaf8c408-4e3a-4fd0-9649-70345f840d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:26.363955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1177/2340] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 09:41:26.364847 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1164/2341] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:41:26.368213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea15720-ac75-496a-aeea-d60ef3572131 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.368766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8c408-4e3a-4fd0-9649-70345f840d38 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.378124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8c408-4e3a-4fd0-9649-70345f840d38 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:26.378341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea15720-ac75-496a-aeea-d60ef3572131 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:26.382957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898af4b0-2888-47a1-84e7-85a7c12d70b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:26.383134 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb094a1-c725-491c-b01b-76820d0c202c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:26.383644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898af4b0-2888-47a1-84e7-85a7c12d70b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.383868 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898af4b0-2888-47a1-84e7-85a7c12d70b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.384044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb094a1-c725-491c-b01b-76820d0c202c admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.384164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898af4b0-2888-47a1-84e7-85a7c12d70b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.384456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb094a1-c725-491c-b01b-76820d0c202c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.384803 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb094a1-c725-491c-b01b-76820d0c202c admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.385497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898af4b0-2888-47a1-84e7-85a7c12d70b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:26.385674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb094a1-c725-491c-b01b-76820d0c202c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:41:26.386052 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb094a1-c725-491c-b01b-76820d0c202c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:26.387423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898af4b0-2888-47a1-84e7-85a7c12d70b0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:26.387738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb094a1-c725-491c-b01b-76820d0c202c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:26.404392 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1165/2342] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 09:41:26.405665 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb094a1-c725-491c-b01b-76820d0c202c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.420280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1178/2343] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:41:26 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:41:26.420479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb094a1-c725-491c-b01b-76820d0c202c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:26.422062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898af4b0-2888-47a1-84e7-85a7c12d70b0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.441524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:26.479684 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac6c16-6b17-4866-8f24-3e4c848043cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:26.480376 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac6c16-6b17-4866-8f24-3e4c848043cc admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.480668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac6c16-6b17-4866-8f24-3e4c848043cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.480946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac6c16-6b17-4866-8f24-3e4c848043cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.481646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac6c16-6b17-4866-8f24-3e4c848043cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:26.483610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac6c16-6b17-4866-8f24-3e4c848043cc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:26.488168 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898af4b0-2888-47a1-84e7-85a7c12d70b0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:26.492839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d036aa8-3338-48f3-a224-dcfab5c99894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:26.493504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d036aa8-3338-48f3-a224-dcfab5c99894 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.493780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d036aa8-3338-48f3-a224-dcfab5c99894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.494050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d036aa8-3338-48f3-a224-dcfab5c99894 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.494737 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d036aa8-3338-48f3-a224-dcfab5c99894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:26.496690 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d036aa8-3338-48f3-a224-dcfab5c99894 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:26.515764 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1166/2344] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:41:26 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:41:26.516823 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac6c16-6b17-4866-8f24-3e4c848043cc admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.520202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1179/2345] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:41:26 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:41:26.521161 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d036aa8-3338-48f3-a224-dcfab5c99894 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.522336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:26.532227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d036aa8-3338-48f3-a224-dcfab5c99894 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:26.586722 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac6c16-6b17-4866-8f24-3e4c848043cc admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:26.591907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa5f17-12e8-41f5-a1ab-208e5bde7102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:26.592599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa5f17-12e8-41f5-a1ab-208e5bde7102 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.592828 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cacff-6692-425f-9839-b5ac4573004f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:26.592960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa5f17-12e8-41f5-a1ab-208e5bde7102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.593183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa5f17-12e8-41f5-a1ab-208e5bde7102 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.593557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cacff-6692-425f-9839-b5ac4573004f admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.593849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cacff-6692-425f-9839-b5ac4573004f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.593964 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa5f17-12e8-41f5-a1ab-208e5bde7102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:26.594129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270cacff-6692-425f-9839-b5ac4573004f admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.594900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cacff-6692-425f-9839-b5ac4573004f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:26.595829 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa5f17-12e8-41f5-a1ab-208e5bde7102 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:26.596659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270cacff-6692-425f-9839-b5ac4573004f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:26.613372 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1180/2346] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:41:26.614482 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cacff-6692-425f-9839-b5ac4573004f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.619843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1167/2347] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:41:26 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:41:26.624178 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa5f17-12e8-41f5-a1ab-208e5bde7102 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.626092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270cacff-6692-425f-9839-b5ac4573004f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:26.632248 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2a4c6-ef8b-478d-b00e-0734afba2f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:26.632917 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2a4c6-ef8b-478d-b00e-0734afba2f82 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.633196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2a4c6-ef8b-478d-b00e-0734afba2f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.633516 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2a4c6-ef8b-478d-b00e-0734afba2f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.634275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2a4c6-ef8b-478d-b00e-0734afba2f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:26.635538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa5f17-12e8-41f5-a1ab-208e5bde7102 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:26.635959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2a4c6-ef8b-478d-b00e-0734afba2f82 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:26.640475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643d2b9-4ffd-4992-9f13-60aacbd230b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:26.641132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643d2b9-4ffd-4992-9f13-60aacbd230b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.641423 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643d2b9-4ffd-4992-9f13-60aacbd230b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.641702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643d2b9-4ffd-4992-9f13-60aacbd230b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.642455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643d2b9-4ffd-4992-9f13-60aacbd230b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:26.644093 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643d2b9-4ffd-4992-9f13-60aacbd230b4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:26.646434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1181/2348] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:41:26.651344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2a4c6-ef8b-478d-b00e-0734afba2f82 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.655156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1168/2349] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:41:26.660326 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643d2b9-4ffd-4992-9f13-60aacbd230b4 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.661716 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2a4c6-ef8b-478d-b00e-0734afba2f82 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:26.667468 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694a433-b9c9-4ae0-8418-ea3fe6a1536d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:26.668401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694a433-b9c9-4ae0-8418-ea3fe6a1536d admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.668690 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694a433-b9c9-4ae0-8418-ea3fe6a1536d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.669044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694a433-b9c9-4ae0-8418-ea3fe6a1536d admin admin] PATH_INFO: `/v3/projects/fc03aebd6a584eafb204f3e466aea0a7/users/63665244e5764abdbe7c26c6731cbd00/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.671375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643d2b9-4ffd-4992-9f13-60aacbd230b4 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:26.675952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3fe71-bcbb-4da6-8d74-d996e74487ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:26.676608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3fe71-bcbb-4da6-8d74-d996e74487ce admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.676888 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3fe71-bcbb-4da6-8d74-d996e74487ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.677163 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3fe71-bcbb-4da6-8d74-d996e74487ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.677892 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3fe71-bcbb-4da6-8d74-d996e74487ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:26.679517 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd3fe71-bcbb-4da6-8d74-d996e74487ce admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:26.686768 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694a433-b9c9-4ae0-8418-ea3fe6a1536d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc03aebd6a584eafb204f3e466aea0a7, user_id=63665244e5764abdbe7c26c6731cbd00, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:26.690173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694a433-b9c9-4ae0-8418-ea3fe6a1536d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:26.691639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1169/2350] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:41:26.697707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3fe71-bcbb-4da6-8d74-d996e74487ce admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.705644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1182/2351] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:41:26 2022] PUT /identity/v3/projects/fc03aebd6a584eafb204f3e466aea0a7/users/63665244e5764abdbe7c26c6731cbd00/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:41:26.707828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3fe71-bcbb-4da6-8d74-d996e74487ce admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:26.722061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef90ff-b988-4d2a-8e90-ea7de96a9d26 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.722377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef90ff-b988-4d2a-8e90-ea7de96a9d26 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.722658 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef90ff-b988-4d2a-8e90-ea7de96a9d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.751315 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e9ef90ff-b988-4d2a-8e90-ea7de96a9d26 None None] MFA Rules not processed for user `63665244e5764abdbe7c26c6731cbd00`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:26.752572 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e9ef90ff-b988-4d2a-8e90-ea7de96a9d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:26.771765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389791a3-03dd-4998-a5d0-57dbe1f1ae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:26.772686 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389791a3-03dd-4998-a5d0-57dbe1f1ae39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.772973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389791a3-03dd-4998-a5d0-57dbe1f1ae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.773273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389791a3-03dd-4998-a5d0-57dbe1f1ae39 admin admin] PATH_INFO: `/v3/projects/1bc0608eb848440ba381d11bb5ef56ea/users/c88c3fce08f64bc3aff957ed842054e5/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.788727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389791a3-03dd-4998-a5d0-57dbe1f1ae39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc0608eb848440ba381d11bb5ef56ea, user_id=c88c3fce08f64bc3aff957ed842054e5, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:26.792812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389791a3-03dd-4998-a5d0-57dbe1f1ae39 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:26.807139 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1170/2352] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:41:26 2022] PUT /identity/v3/projects/1bc0608eb848440ba381d11bb5ef56ea/users/c88c3fce08f64bc3aff957ed842054e5/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:41:26.817243 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a724f-116d-4b1a-b490-2493c8ff7136 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.817537 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a724f-116d-4b1a-b490-2493c8ff7136 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.817702 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:26.817895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a724f-116d-4b1a-b490-2493c8ff7136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.824386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1183/2353] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:41:26.828230 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef90ff-b988-4d2a-8e90-ea7de96a9d26 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.839697 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef90ff-b988-4d2a-8e90-ea7de96a9d26 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:26.849770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-057a724f-116d-4b1a-b490-2493c8ff7136 None None] MFA Rules not processed for user `c88c3fce08f64bc3aff957ed842054e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:26.850980 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-057a724f-116d-4b1a-b490-2493c8ff7136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:26.898045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:26.899630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e656b5-0534-42ba-bc5d-387962189e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:26.900337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e656b5-0534-42ba-bc5d-387962189e6e admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:26.900656 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e656b5-0534-42ba-bc5d-387962189e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:26.900971 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e656b5-0534-42ba-bc5d-387962189e6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:26.901757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e656b5-0534-42ba-bc5d-387962189e6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:26.903716 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e656b5-0534-42ba-bc5d-387962189e6e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:26.923544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1171/2354] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:41:26.934384 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1184/2355] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:41:26 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:41:26.934575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a724f-116d-4b1a-b490-2493c8ff7136 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.935329 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e656b5-0534-42ba-bc5d-387962189e6e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:26.936048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-057a724f-116d-4b1a-b490-2493c8ff7136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:26.994069 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a724f-116d-4b1a-b490-2493c8ff7136 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:27.000048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e656b5-0534-42ba-bc5d-387962189e6e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:27.004820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1e2b3-8225-4cca-a9ab-482d496a3278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:27.005490 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1e2b3-8225-4cca-a9ab-482d496a3278 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:27.005778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1e2b3-8225-4cca-a9ab-482d496a3278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:27.006064 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1e2b3-8225-4cca-a9ab-482d496a3278 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:27.006757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1e2b3-8225-4cca-a9ab-482d496a3278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:27.009368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1e2b3-8225-4cca-a9ab-482d496a3278 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:27.029768 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1185/2356] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:41:26 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:41:27.033557 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1e2b3-8225-4cca-a9ab-482d496a3278 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:27.043393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1e2b3-8225-4cca-a9ab-482d496a3278 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:27.045435 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aab81a-0ca9-4da7-bea5-7cc39e755197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:27.046355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aab81a-0ca9-4da7-bea5-7cc39e755197 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:27.046616 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aab81a-0ca9-4da7-bea5-7cc39e755197 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:27.047004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aab81a-0ca9-4da7-bea5-7cc39e755197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:27.048929 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c9772-f44d-4ebd-a390-a6f969c65a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:27.049588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c9772-f44d-4ebd-a390-a6f969c65a4c admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:27.049908 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c9772-f44d-4ebd-a390-a6f969c65a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:27.050184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c9772-f44d-4ebd-a390-a6f969c65a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:27.050938 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c9772-f44d-4ebd-a390-a6f969c65a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:27.052759 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c9772-f44d-4ebd-a390-a6f969c65a4c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:27.063533 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1186/2357] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:41:27.067717 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c9772-f44d-4ebd-a390-a6f969c65a4c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:27.078196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c9772-f44d-4ebd-a390-a6f969c65a4c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:27.083098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ca60a-5686-41fa-9f78-08925a072ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:27.083992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ca60a-5686-41fa-9f78-08925a072ce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:27.084301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ca60a-5686-41fa-9f78-08925a072ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:27.084561 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ca60a-5686-41fa-9f78-08925a072ce0 admin admin] PATH_INFO: `/v3/projects/4386eba09e4b4dd5be79a0d89e4c5fa9/users/bb083fbd120e499591abc3140188db18/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:27.100263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ca60a-5686-41fa-9f78-08925a072ce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4386eba09e4b4dd5be79a0d89e4c5fa9, user_id=bb083fbd120e499591abc3140188db18, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:27.103199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ca60a-5686-41fa-9f78-08925a072ce0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:27.105073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aab81a-0ca9-4da7-bea5-7cc39e755197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:27.106714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aab81a-0ca9-4da7-bea5-7cc39e755197 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:27.116274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1187/2358] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:41:27 2022] PUT /identity/v3/projects/4386eba09e4b4dd5be79a0d89e4c5fa9/users/bb083fbd120e499591abc3140188db18/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:41:27.116928 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1172/2359] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:41:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:41:27.121326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ca60a-5686-41fa-9f78-08925a072ce0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:27.164472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:27.188683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ca60a-5686-41fa-9f78-08925a072ce0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:27.193575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93842e9-949f-4149-943f-a81c536ecbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:27.194242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93842e9-949f-4149-943f-a81c536ecbad admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:27.194534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93842e9-949f-4149-943f-a81c536ecbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:27.194813 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93842e9-949f-4149-943f-a81c536ecbad admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:27.195575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93842e9-949f-4149-943f-a81c536ecbad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:27.197249 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93842e9-949f-4149-943f-a81c536ecbad admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:27.207882 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1188/2360] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:41:27.212386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aab81a-0ca9-4da7-bea5-7cc39e755197 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:27.223090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aab81a-0ca9-4da7-bea5-7cc39e755197 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:27.228742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f64d61-cac0-4ad8-b351-1cbe781139a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:27.229624 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f64d61-cac0-4ad8-b351-1cbe781139a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:27.229906 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f64d61-cac0-4ad8-b351-1cbe781139a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:27.230176 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f64d61-cac0-4ad8-b351-1cbe781139a1 admin admin] PATH_INFO: `/v3/projects/4386eba09e4b4dd5be79a0d89e4c5fa9/users/bb083fbd120e499591abc3140188db18/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:27.232188 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f64d61-cac0-4ad8-b351-1cbe781139a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4386eba09e4b4dd5be79a0d89e4c5fa9, user_id=bb083fbd120e499591abc3140188db18, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:27.235559 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f64d61-cac0-4ad8-b351-1cbe781139a1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:27.247223 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1173/2361] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:41:27 2022] PUT /identity/v3/projects/4386eba09e4b4dd5be79a0d89e4c5fa9/users/bb083fbd120e499591abc3140188db18/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:41:27.257676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6ab3f-8ee9-4844-ae25-537e8631acfd None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:27.257964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6ab3f-8ee9-4844-ae25-537e8631acfd None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:27.258243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6ab3f-8ee9-4844-ae25-537e8631acfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:27.287589 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-21c6ab3f-8ee9-4844-ae25-537e8631acfd None None] MFA Rules not processed for user `bb083fbd120e499591abc3140188db18`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:27.288844 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-21c6ab3f-8ee9-4844-ae25-537e8631acfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:27.363982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1189/2362] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:27 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:41:27.481426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f64d61-cac0-4ad8-b351-1cbe781139a1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:27.527518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:27.538962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f64d61-cac0-4ad8-b351-1cbe781139a1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:27.542611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e028a87-88cf-4626-8179-3ca17850780c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:27.543246 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e028a87-88cf-4626-8179-3ca17850780c service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:27.543526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e028a87-88cf-4626-8179-3ca17850780c service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:27.543797 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e028a87-88cf-4626-8179-3ca17850780c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:27.553328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e028a87-88cf-4626-8179-3ca17850780c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:27.554770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e028a87-88cf-4626-8179-3ca17850780c service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:27.564805 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1174/2363] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:41:27 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:41:27.667839 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6ab3f-8ee9-4844-ae25-537e8631acfd None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:27.679941 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6ab3f-8ee9-4844-ae25-537e8631acfd None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:27.684150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976f25a-9099-4541-ae19-f8753a4e1883 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:27.684896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976f25a-9099-4541-ae19-f8753a4e1883 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:27.685328 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976f25a-9099-4541-ae19-f8753a4e1883 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:27.685957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976f25a-9099-4541-ae19-f8753a4e1883 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:27.747440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976f25a-9099-4541-ae19-f8753a4e1883 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:27.748933 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976f25a-9099-4541-ae19-f8753a4e1883 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:27.757953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1190/2364] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:41:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:41:30.485695 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e028a87-88cf-4626-8179-3ca17850780c service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:30.551419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e028a87-88cf-4626-8179-3ca17850780c service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:30.556452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d36d4-6548-43e5-81ca-3f40b772edac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:30.557653 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d36d4-6548-43e5-81ca-3f40b772edac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:30.558851 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d36d4-6548-43e5-81ca-3f40b772edac admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:30.558851 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d36d4-6548-43e5-81ca-3f40b772edac admin admin] PATH_INFO: `/v3/users/63665244e5764abdbe7c26c6731cbd00` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:30.559995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d36d4-6548-43e5-81ca-3f40b772edac admin admin] RBAC: Authorizing `identity:delete_user(user_id=63665244e5764abdbe7c26c6731cbd00)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:30.562007 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d36d4-6548-43e5-81ca-3f40b772edac admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:30.615100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-536d36d4-6548-43e5-81ca-3f40b772edac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63665244e5764abdbe7c26c6731cbd00'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:30.623420 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-536d36d4-6548-43e5-81ca-3f40b772edac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63665244e5764abdbe7c26c6731cbd00'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:30.630769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-536d36d4-6548-43e5-81ca-3f40b772edac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63665244e5764abdbe7c26c6731cbd00'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:30.646121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-536d36d4-6548-43e5-81ca-3f40b772edac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63665244e5764abdbe7c26c6731cbd00'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:30.646911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-536d36d4-6548-43e5-81ca-3f40b772edac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63665244e5764abdbe7c26c6731cbd00'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:30.653875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1175/2365] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:41:30 2022] DELETE /identity/v3/users/63665244e5764abdbe7c26c6731cbd00 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:41:30.658952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbeb46f-73ef-49c3-bad5-651974be4e0d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:30.659599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbeb46f-73ef-49c3-bad5-651974be4e0d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:30.659799 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbeb46f-73ef-49c3-bad5-651974be4e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:30.678018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6bbeb46f-73ef-49c3-bad5-651974be4e0d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:30.679244 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6bbeb46f-73ef-49c3-bad5-651974be4e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:30.744976 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:30.755648 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1191/2366] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:41:30.768021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d36d4-6548-43e5-81ca-3f40b772edac admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:30.769439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-536d36d4-6548-43e5-81ca-3f40b772edac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:30.830359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d36d4-6548-43e5-81ca-3f40b772edac admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:30.833982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eda0db-aaaf-46d5-a7d3-cab64a4d9a02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:30.834616 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eda0db-aaaf-46d5-a7d3-cab64a4d9a02 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:30.834902 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eda0db-aaaf-46d5-a7d3-cab64a4d9a02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:30.835169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eda0db-aaaf-46d5-a7d3-cab64a4d9a02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:30.844157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eda0db-aaaf-46d5-a7d3-cab64a4d9a02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:30.845600 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eda0db-aaaf-46d5-a7d3-cab64a4d9a02 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:30.855141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1176/2367] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:41:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:41:31.129575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbeb46f-73ef-49c3-bad5-651974be4e0d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:31.131355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6bbeb46f-73ef-49c3-bad5-651974be4e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:31.143644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbeb46f-73ef-49c3-bad5-651974be4e0d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:31.148250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca351ef-4695-480e-bdea-5dec1661e22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:31.149228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca351ef-4695-480e-bdea-5dec1661e22c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:31.149553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca351ef-4695-480e-bdea-5dec1661e22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:31.149831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca351ef-4695-480e-bdea-5dec1661e22c admin admin] PATH_INFO: `/v3/projects/fc03aebd6a584eafb204f3e466aea0a7` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:31.151181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca351ef-4695-480e-bdea-5dec1661e22c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc03aebd6a584eafb204f3e466aea0a7)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:31.153153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca351ef-4695-480e-bdea-5dec1661e22c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:31.201054 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1ca351ef-4695-480e-bdea-5dec1661e22c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc03aebd6a584eafb204f3e466aea0a7', 'request_id': 'req-1ca351ef-4695-480e-bdea-5dec1661e22c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:31.206525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1ca351ef-4695-480e-bdea-5dec1661e22c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc03aebd6a584eafb204f3e466aea0a7', 'request_id': 'req-1ca351ef-4695-480e-bdea-5dec1661e22c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:31.213022 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1ca351ef-4695-480e-bdea-5dec1661e22c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc03aebd6a584eafb204f3e466aea0a7', 'request_id': 'req-1ca351ef-4695-480e-bdea-5dec1661e22c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:31.225295 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1ca351ef-4695-480e-bdea-5dec1661e22c admin admin] The token cache is being invalidate because project fc03aebd6a584eafb204f3e466aea0a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:41:31.225584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1ca351ef-4695-480e-bdea-5dec1661e22c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:31.226221 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1ca351ef-4695-480e-bdea-5dec1661e22c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:31.228314 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1192/2368] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:41:31 2022] DELETE /identity/v3/projects/fc03aebd6a584eafb204f3e466aea0a7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:41:31.232308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eda0db-aaaf-46d5-a7d3-cab64a4d9a02 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:31.242206 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eda0db-aaaf-46d5-a7d3-cab64a4d9a02 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:31.246483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:31.302314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384cb2dd-05c2-4a0e-9689-584af48e44c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:31.303178 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cb2dd-05c2-4a0e-9689-584af48e44c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:31.303455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cb2dd-05c2-4a0e-9689-584af48e44c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:31.303727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384cb2dd-05c2-4a0e-9689-584af48e44c7 admin admin] PATH_INFO: `/v3/users/bb083fbd120e499591abc3140188db18` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:31.304996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384cb2dd-05c2-4a0e-9689-584af48e44c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb083fbd120e499591abc3140188db18)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:31.307848 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384cb2dd-05c2-4a0e-9689-584af48e44c7 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:31.355976 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-384cb2dd-05c2-4a0e-9689-584af48e44c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb083fbd120e499591abc3140188db18'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:31.363241 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-384cb2dd-05c2-4a0e-9689-584af48e44c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb083fbd120e499591abc3140188db18'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:31.373054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-384cb2dd-05c2-4a0e-9689-584af48e44c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb083fbd120e499591abc3140188db18'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:31.385745 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-384cb2dd-05c2-4a0e-9689-584af48e44c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb083fbd120e499591abc3140188db18'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:31.386441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-384cb2dd-05c2-4a0e-9689-584af48e44c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb083fbd120e499591abc3140188db18'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:31.392448 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1177/2369] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:41:31 2022] DELETE /identity/v3/users/bb083fbd120e499591abc3140188db18 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:41:31.691842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca351ef-4695-480e-bdea-5dec1661e22c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:31.693677 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1ca351ef-4695-480e-bdea-5dec1661e22c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:31.717179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca351ef-4695-480e-bdea-5dec1661e22c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:31.775261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a476dd1-a8c1-4793-af90-2ecb0ecee382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:31.776179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a476dd1-a8c1-4793-af90-2ecb0ecee382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:31.776467 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a476dd1-a8c1-4793-af90-2ecb0ecee382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:31.776745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a476dd1-a8c1-4793-af90-2ecb0ecee382 admin admin] PATH_INFO: `/v3/projects/4386eba09e4b4dd5be79a0d89e4c5fa9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:31.778092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a476dd1-a8c1-4793-af90-2ecb0ecee382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4386eba09e4b4dd5be79a0d89e4c5fa9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:31.780082 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a476dd1-a8c1-4793-af90-2ecb0ecee382 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:31.839204 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-2a476dd1-a8c1-4793-af90-2ecb0ecee382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4386eba09e4b4dd5be79a0d89e4c5fa9', 'request_id': 'req-2a476dd1-a8c1-4793-af90-2ecb0ecee382'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:31.845686 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-2a476dd1-a8c1-4793-af90-2ecb0ecee382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4386eba09e4b4dd5be79a0d89e4c5fa9', 'request_id': 'req-2a476dd1-a8c1-4793-af90-2ecb0ecee382'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:31.852089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-2a476dd1-a8c1-4793-af90-2ecb0ecee382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4386eba09e4b4dd5be79a0d89e4c5fa9', 'request_id': 'req-2a476dd1-a8c1-4793-af90-2ecb0ecee382'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:31.864888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-2a476dd1-a8c1-4793-af90-2ecb0ecee382 admin admin] The token cache is being invalidate because project 4386eba09e4b4dd5be79a0d89e4c5fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:41:31.865184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-2a476dd1-a8c1-4793-af90-2ecb0ecee382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:31.865806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-2a476dd1-a8c1-4793-af90-2ecb0ecee382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:31.867377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1193/2370] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:41:31 2022] DELETE /identity/v3/projects/4386eba09e4b4dd5be79a0d89e4c5fa9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:41:31.873145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e475ff3-ab56-4601-9e6e-592b52f022fa None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:31.873667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e475ff3-ab56-4601-9e6e-592b52f022fa None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:31.873802 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e475ff3-ab56-4601-9e6e-592b52f022fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:31.892293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0e475ff3-ab56-4601-9e6e-592b52f022fa None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:31.893531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0e475ff3-ab56-4601-9e6e-592b52f022fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:31.951809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1178/2371] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:41:31.959594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c050-45da-430c-bed5-a3687cd28fce None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:31.959891 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c050-45da-430c-bed5-a3687cd28fce None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:31.960207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c050-45da-430c-bed5-a3687cd28fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:31.979970 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-2c02c050-45da-430c-bed5-a3687cd28fce None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:31.981192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2c02c050-45da-430c-bed5-a3687cd28fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:31.989189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1194/2372] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:41:31.993469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e475ff3-ab56-4601-9e6e-592b52f022fa None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:32.003103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e475ff3-ab56-4601-9e6e-592b52f022fa None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:32.007689 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c47cb-a359-43dd-8399-017440adbfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:32.008929 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c47cb-a359-43dd-8399-017440adbfdb admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:32.009170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c47cb-a359-43dd-8399-017440adbfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:32.009447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c47cb-a359-43dd-8399-017440adbfdb admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:32.010320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c47cb-a359-43dd-8399-017440adbfdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:41:32.010628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c47cb-a359-43dd-8399-017440adbfdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:32.012278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c47cb-a359-43dd-8399-017440adbfdb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:32.024001 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1179/2373] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:41:32.029020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02c050-45da-430c-bed5-a3687cd28fce None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:32.034577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:32.038618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02c050-45da-430c-bed5-a3687cd28fce None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:32.044710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1cea1-3384-4831-bab5-18bde1bb1424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:32.045855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1cea1-3384-4831-bab5-18bde1bb1424 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:32.046407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1cea1-3384-4831-bab5-18bde1bb1424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:32.046876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1cea1-3384-4831-bab5-18bde1bb1424 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:32.047898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1cea1-3384-4831-bab5-18bde1bb1424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:32.049858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1cea1-3384-4831-bab5-18bde1bb1424 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:32.076499 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1195/2374] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:41:32 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:41:32.080839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c47cb-a359-43dd-8399-017440adbfdb admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:32.090781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c47cb-a359-43dd-8399-017440adbfdb admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:32.099430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:32.151001 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede6be2-3041-4489-8a54-05ece892ad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:32.151834 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede6be2-3041-4489-8a54-05ece892ad7b admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:32.152149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede6be2-3041-4489-8a54-05ece892ad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:32.152427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede6be2-3041-4489-8a54-05ece892ad7b admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:32.153127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede6be2-3041-4489-8a54-05ece892ad7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:32.155130 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede6be2-3041-4489-8a54-05ece892ad7b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:32.176141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1180/2375] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:41:32 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:41:32.181057 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1cea1-3384-4831-bab5-18bde1bb1424 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:32.191429 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1cea1-3384-4831-bab5-18bde1bb1424 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:32.196751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcd9cd-6df8-4b70-bce0-44a2235fa358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:32.197364 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcd9cd-6df8-4b70-bce0-44a2235fa358 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:32.197649 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcd9cd-6df8-4b70-bce0-44a2235fa358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:32.197931 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcd9cd-6df8-4b70-bce0-44a2235fa358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:32.198683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcd9cd-6df8-4b70-bce0-44a2235fa358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:32.200332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcd9cd-6df8-4b70-bce0-44a2235fa358 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:32.210718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1196/2376] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:41:32.215370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede6be2-3041-4489-8a54-05ece892ad7b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:32.225492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede6be2-3041-4489-8a54-05ece892ad7b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:32.230852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83005705-c2a0-4e37-a747-1edda22886a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:32.231511 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83005705-c2a0-4e37-a747-1edda22886a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:32.231792 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83005705-c2a0-4e37-a747-1edda22886a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:32.232104 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83005705-c2a0-4e37-a747-1edda22886a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:32.232859 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83005705-c2a0-4e37-a747-1edda22886a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:32.234473 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83005705-c2a0-4e37-a747-1edda22886a1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:32.245228 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1181/2377] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:41:32.249385 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcd9cd-6df8-4b70-bce0-44a2235fa358 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:32.260128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcd9cd-6df8-4b70-bce0-44a2235fa358 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:32.265555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aedf0b-0c71-4ee5-b611-14b90ed77e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:32.266423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aedf0b-0c71-4ee5-b611-14b90ed77e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:32.266701 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aedf0b-0c71-4ee5-b611-14b90ed77e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:32.266972 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aedf0b-0c71-4ee5-b611-14b90ed77e2e admin admin] PATH_INFO: `/v3/projects/bf289a717df44c8d81e236e1c0a74cc1/users/d69df5a1f1b642d8b417739563bdfc90/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:32.282996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aedf0b-0c71-4ee5-b611-14b90ed77e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf289a717df44c8d81e236e1c0a74cc1, user_id=d69df5a1f1b642d8b417739563bdfc90, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:32.285682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aedf0b-0c71-4ee5-b611-14b90ed77e2e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:32.302208 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1197/2378] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:41:32 2022] PUT /identity/v3/projects/bf289a717df44c8d81e236e1c0a74cc1/users/d69df5a1f1b642d8b417739563bdfc90/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:41:32.315095 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cd9f5-c549-48b1-8127-2056f9a1c3ae None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:32.315386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cd9f5-c549-48b1-8127-2056f9a1c3ae None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:32.315667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cd9f5-c549-48b1-8127-2056f9a1c3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:32.342574 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3a6cd9f5-c549-48b1-8127-2056f9a1c3ae None None] MFA Rules not processed for user `d69df5a1f1b642d8b417739563bdfc90`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:32.343798 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3a6cd9f5-c549-48b1-8127-2056f9a1c3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:32.408410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1182/2379] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:41:32.412758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aedf0b-0c71-4ee5-b611-14b90ed77e2e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:32.475545 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aedf0b-0c71-4ee5-b611-14b90ed77e2e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:32.481538 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ead63-5c2d-4307-9c51-4b56ca529d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:32.482220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ead63-5c2d-4307-9c51-4b56ca529d11 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:32.482505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ead63-5c2d-4307-9c51-4b56ca529d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:32.482780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ead63-5c2d-4307-9c51-4b56ca529d11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:32.483477 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ead63-5c2d-4307-9c51-4b56ca529d11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:32.485440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ead63-5c2d-4307-9c51-4b56ca529d11 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:32.511773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1198/2380] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:41:32 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:41:32.515930 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cd9f5-c549-48b1-8127-2056f9a1c3ae None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:32.526812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cd9f5-c549-48b1-8127-2056f9a1c3ae None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:32.584759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07178d78-8d5e-4d04-9f95-7fc90aa88c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:32.585430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07178d78-8d5e-4d04-9f95-7fc90aa88c6e admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:32.585717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07178d78-8d5e-4d04-9f95-7fc90aa88c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:32.585998 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07178d78-8d5e-4d04-9f95-7fc90aa88c6e admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:32.586773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07178d78-8d5e-4d04-9f95-7fc90aa88c6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:32.589102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07178d78-8d5e-4d04-9f95-7fc90aa88c6e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:32.609320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1183/2381] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:41:32 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:41:32.613156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ead63-5c2d-4307-9c51-4b56ca529d11 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:32.624740 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ead63-5c2d-4307-9c51-4b56ca529d11 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:32.631676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154dbbb-c611-4c9b-a9df-952c7a3ca9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:32.631676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154dbbb-c611-4c9b-a9df-952c7a3ca9f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:32.631676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154dbbb-c611-4c9b-a9df-952c7a3ca9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:32.631676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154dbbb-c611-4c9b-a9df-952c7a3ca9f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:32.632329 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154dbbb-c611-4c9b-a9df-952c7a3ca9f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:32.633985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154dbbb-c611-4c9b-a9df-952c7a3ca9f5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:32.649509 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1199/2382] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:41:32.653410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07178d78-8d5e-4d04-9f95-7fc90aa88c6e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:32.667483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07178d78-8d5e-4d04-9f95-7fc90aa88c6e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:32.672473 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b695a6b-ac6d-4cd8-922e-24eaf3f0e3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:32.673348 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b695a6b-ac6d-4cd8-922e-24eaf3f0e3f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:32.673763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b695a6b-ac6d-4cd8-922e-24eaf3f0e3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:32.673896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b695a6b-ac6d-4cd8-922e-24eaf3f0e3f8 admin admin] PATH_INFO: `/v3/projects/98cbe00d956246dfbb9030000bb8916e/users/cb9b947c78c746d2924efadb14b65a55/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:32.683367 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:32.693292 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b695a6b-ac6d-4cd8-922e-24eaf3f0e3f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98cbe00d956246dfbb9030000bb8916e, user_id=cb9b947c78c746d2924efadb14b65a55, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:32.696948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b695a6b-ac6d-4cd8-922e-24eaf3f0e3f8 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:32.711476 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1184/2383] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:41:32 2022] PUT /identity/v3/projects/98cbe00d956246dfbb9030000bb8916e/users/cb9b947c78c746d2924efadb14b65a55/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:41:32.716884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154dbbb-c611-4c9b-a9df-952c7a3ca9f5 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:32.729129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154dbbb-c611-4c9b-a9df-952c7a3ca9f5 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:32.787840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b52ba3-64fe-4477-8040-bf462b78cb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:32.788535 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b52ba3-64fe-4477-8040-bf462b78cb91 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:32.788815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b52ba3-64fe-4477-8040-bf462b78cb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:32.789085 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b52ba3-64fe-4477-8040-bf462b78cb91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:32.790553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b52ba3-64fe-4477-8040-bf462b78cb91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:32.792164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b52ba3-64fe-4477-8040-bf462b78cb91 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:32.802243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1200/2384] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:41:32.806757 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b695a6b-ac6d-4cd8-922e-24eaf3f0e3f8 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:32.817086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b695a6b-ac6d-4cd8-922e-24eaf3f0e3f8 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:32.821758 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb7455-c31e-4c5a-9485-90db830d4e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:32.822632 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb7455-c31e-4c5a-9485-90db830d4e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:32.822916 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb7455-c31e-4c5a-9485-90db830d4e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:32.823193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb7455-c31e-4c5a-9485-90db830d4e61 admin admin] PATH_INFO: `/v3/projects/98cbe00d956246dfbb9030000bb8916e/users/cb9b947c78c746d2924efadb14b65a55/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:32.825180 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb7455-c31e-4c5a-9485-90db830d4e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98cbe00d956246dfbb9030000bb8916e, user_id=cb9b947c78c746d2924efadb14b65a55, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:32.827846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb7455-c31e-4c5a-9485-90db830d4e61 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:32.839226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1185/2385] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:41:32 2022] PUT /identity/v3/projects/98cbe00d956246dfbb9030000bb8916e/users/cb9b947c78c746d2924efadb14b65a55/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:41:32.849453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae1979-07bb-47d5-a431-a535d5296316 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:32.849745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae1979-07bb-47d5-a431-a535d5296316 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:32.850019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae1979-07bb-47d5-a431-a535d5296316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:32.878969 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a9ae1979-07bb-47d5-a431-a535d5296316 None None] MFA Rules not processed for user `cb9b947c78c746d2924efadb14b65a55`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:32.880221 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a9ae1979-07bb-47d5-a431-a535d5296316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:32.886089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:32.949935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1201/2386] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:41:32.960377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb7455-c31e-4c5a-9485-90db830d4e61 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:32.962149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f4bb7455-c31e-4c5a-9485-90db830d4e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:33.018199 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb7455-c31e-4c5a-9485-90db830d4e61 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:33.021825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240282f-1951-421f-b23d-1c15f4195c13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:33.022454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240282f-1951-421f-b23d-1c15f4195c13 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:33.022733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240282f-1951-421f-b23d-1c15f4195c13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:33.023001 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240282f-1951-421f-b23d-1c15f4195c13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:33.032094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240282f-1951-421f-b23d-1c15f4195c13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:33.033530 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240282f-1951-421f-b23d-1c15f4195c13 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:33.042849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1186/2387] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:41:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:41:33.604295 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae1979-07bb-47d5-a431-a535d5296316 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:33.669060 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae1979-07bb-47d5-a431-a535d5296316 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:33.675854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa289f-262f-4f68-abf3-514ba57c60de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:33.677326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa289f-262f-4f68-abf3-514ba57c60de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:33.677807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa289f-262f-4f68-abf3-514ba57c60de admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:33.678195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa289f-262f-4f68-abf3-514ba57c60de admin admin] PATH_INFO: `/v3/users/d69df5a1f1b642d8b417739563bdfc90` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:33.679661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa289f-262f-4f68-abf3-514ba57c60de admin admin] RBAC: Authorizing `identity:delete_user(user_id=d69df5a1f1b642d8b417739563bdfc90)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:33.682638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa289f-262f-4f68-abf3-514ba57c60de admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:33.735744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b5fa289f-262f-4f68-abf3-514ba57c60de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd69df5a1f1b642d8b417739563bdfc90'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:33.736491 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b5fa289f-262f-4f68-abf3-514ba57c60de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd69df5a1f1b642d8b417739563bdfc90'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:33.745803 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b5fa289f-262f-4f68-abf3-514ba57c60de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd69df5a1f1b642d8b417739563bdfc90'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:33.759960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b5fa289f-262f-4f68-abf3-514ba57c60de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd69df5a1f1b642d8b417739563bdfc90'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:33.768207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b5fa289f-262f-4f68-abf3-514ba57c60de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd69df5a1f1b642d8b417739563bdfc90'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:33.775348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1202/2388] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:41:33 2022] DELETE /identity/v3/users/d69df5a1f1b642d8b417739563bdfc90 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:41:33.780276 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34033074-f771-4b33-b14f-32d003a1a9ed None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:33.780864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34033074-f771-4b33-b14f-32d003a1a9ed None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:33.781365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34033074-f771-4b33-b14f-32d003a1a9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:33.801926 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-34033074-f771-4b33-b14f-32d003a1a9ed None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:33.803151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-34033074-f771-4b33-b14f-32d003a1a9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:33.863508 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1187/2389] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:41:33.875358 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa289f-262f-4f68-abf3-514ba57c60de admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:33.876870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b5fa289f-262f-4f68-abf3-514ba57c60de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:33.905773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:33.941844 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa289f-262f-4f68-abf3-514ba57c60de admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:33.945553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3afc8-d581-4d8c-8335-ef258e8e5645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:33.946618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3afc8-d581-4d8c-8335-ef258e8e5645 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:33.946949 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3afc8-d581-4d8c-8335-ef258e8e5645 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:33.947167 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3afc8-d581-4d8c-8335-ef258e8e5645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:33.956330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3afc8-d581-4d8c-8335-ef258e8e5645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:33.957714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3afc8-d581-4d8c-8335-ef258e8e5645 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:33.967259 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1203/2390] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:41:34.234207 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34033074-f771-4b33-b14f-32d003a1a9ed None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:34.235722 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-34033074-f771-4b33-b14f-32d003a1a9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:34.248763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34033074-f771-4b33-b14f-32d003a1a9ed None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:34.253707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:34.254971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:34.255532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:34.255814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3 admin admin] PATH_INFO: `/v3/projects/bf289a717df44c8d81e236e1c0a74cc1` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:34.257086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf289a717df44c8d81e236e1c0a74cc1)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:34.259198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:34.309140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf289a717df44c8d81e236e1c0a74cc1', 'request_id': 'req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:34.315294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf289a717df44c8d81e236e1c0a74cc1', 'request_id': 'req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:34.323574 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf289a717df44c8d81e236e1c0a74cc1', 'request_id': 'req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:34.330573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:34.334916 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3 admin admin] The token cache is being invalidate because project bf289a717df44c8d81e236e1c0a74cc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:41:34.335439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:34.336196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:34.338442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1188/2391] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:41:34 2022] DELETE /identity/v3/projects/bf289a717df44c8d81e236e1c0a74cc1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:41:34.348203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3afc8-d581-4d8c-8335-ef258e8e5645 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:34.358670 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3afc8-d581-4d8c-8335-ef258e8e5645 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:34.416062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593326a-7fd1-42d7-8e46-e7fbe25331da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:34.416938 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593326a-7fd1-42d7-8e46-e7fbe25331da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:34.417225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593326a-7fd1-42d7-8e46-e7fbe25331da admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:34.417501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593326a-7fd1-42d7-8e46-e7fbe25331da admin admin] PATH_INFO: `/v3/users/cb9b947c78c746d2924efadb14b65a55` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:34.418693 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593326a-7fd1-42d7-8e46-e7fbe25331da admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb9b947c78c746d2924efadb14b65a55)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:34.420952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593326a-7fd1-42d7-8e46-e7fbe25331da admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:34.469264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e593326a-7fd1-42d7-8e46-e7fbe25331da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb9b947c78c746d2924efadb14b65a55'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:34.469935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e593326a-7fd1-42d7-8e46-e7fbe25331da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb9b947c78c746d2924efadb14b65a55'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:34.477524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e593326a-7fd1-42d7-8e46-e7fbe25331da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb9b947c78c746d2924efadb14b65a55'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:34.491118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e593326a-7fd1-42d7-8e46-e7fbe25331da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb9b947c78c746d2924efadb14b65a55'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:34.498578 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e593326a-7fd1-42d7-8e46-e7fbe25331da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb9b947c78c746d2924efadb14b65a55'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:34.504690 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1204/2392] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:41:34 2022] DELETE /identity/v3/users/cb9b947c78c746d2924efadb14b65a55 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:41:34.794469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:34.796748 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:34.814830 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe4bb4-98f6-405e-aa28-1bed471ef3a3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:34.885714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ac8f7-2323-4ad6-b147-4f1250cc5f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:34.886583 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ac8f7-2323-4ad6-b147-4f1250cc5f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:34.886867 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ac8f7-2323-4ad6-b147-4f1250cc5f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:34.887141 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ac8f7-2323-4ad6-b147-4f1250cc5f57 admin admin] PATH_INFO: `/v3/projects/98cbe00d956246dfbb9030000bb8916e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:34.888363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ac8f7-2323-4ad6-b147-4f1250cc5f57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98cbe00d956246dfbb9030000bb8916e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:34.890306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ac8f7-2323-4ad6-b147-4f1250cc5f57 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:34.937397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-740ac8f7-2323-4ad6-b147-4f1250cc5f57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98cbe00d956246dfbb9030000bb8916e', 'request_id': 'req-740ac8f7-2323-4ad6-b147-4f1250cc5f57'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:34.943853 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-740ac8f7-2323-4ad6-b147-4f1250cc5f57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98cbe00d956246dfbb9030000bb8916e', 'request_id': 'req-740ac8f7-2323-4ad6-b147-4f1250cc5f57'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:34.953380 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-740ac8f7-2323-4ad6-b147-4f1250cc5f57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98cbe00d956246dfbb9030000bb8916e', 'request_id': 'req-740ac8f7-2323-4ad6-b147-4f1250cc5f57'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:34.964984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-740ac8f7-2323-4ad6-b147-4f1250cc5f57 admin admin] The token cache is being invalidate because project 98cbe00d956246dfbb9030000bb8916e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:41:34.965140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-740ac8f7-2323-4ad6-b147-4f1250cc5f57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:34.965794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-740ac8f7-2323-4ad6-b147-4f1250cc5f57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:41:34.967569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1189/2393] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:41:34 2022] DELETE /identity/v3/projects/98cbe00d956246dfbb9030000bb8916e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:41:34.972311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b61c1-dab0-4701-8e9f-e50d317752cb None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:34.972798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b61c1-dab0-4701-8e9f-e50d317752cb None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:34.972901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b61c1-dab0-4701-8e9f-e50d317752cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:34.991782 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f33b61c1-dab0-4701-8e9f-e50d317752cb None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:34.993069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f33b61c1-dab0-4701-8e9f-e50d317752cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:35.035731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:35.056923 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1205/2394] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:41:35.065900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a327bc-b501-4316-b9d3-6f336a5ed6d4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:35.066185 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a327bc-b501-4316-b9d3-6f336a5ed6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:35.066480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a327bc-b501-4316-b9d3-6f336a5ed6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:35.085272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a9a327bc-b501-4316-b9d3-6f336a5ed6d4 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:35.086558 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a9a327bc-b501-4316-b9d3-6f336a5ed6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:35.094578 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1190/2395] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:41:35.098751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b61c1-dab0-4701-8e9f-e50d317752cb None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:35.108650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b61c1-dab0-4701-8e9f-e50d317752cb None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:35.113112 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e54f8c-4cd5-42b7-b56a-d52b15e9e8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:35.113761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e54f8c-4cd5-42b7-b56a-d52b15e9e8cd admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:35.114046 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e54f8c-4cd5-42b7-b56a-d52b15e9e8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:35.114323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e54f8c-4cd5-42b7-b56a-d52b15e9e8cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:35.115047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e54f8c-4cd5-42b7-b56a-d52b15e9e8cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:41:35.115354 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e54f8c-4cd5-42b7-b56a-d52b15e9e8cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:35.117015 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e54f8c-4cd5-42b7-b56a-d52b15e9e8cd admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:35.129102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1206/2396] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 09:41:35.135595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a327bc-b501-4316-b9d3-6f336a5ed6d4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:35.144827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a327bc-b501-4316-b9d3-6f336a5ed6d4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:35.150492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50005b-3bb9-4593-a936-955399ac0259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:35.151143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50005b-3bb9-4593-a936-955399ac0259 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:35.151431 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50005b-3bb9-4593-a936-955399ac0259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:35.151707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50005b-3bb9-4593-a936-955399ac0259 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:35.152433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50005b-3bb9-4593-a936-955399ac0259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:35.154425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50005b-3bb9-4593-a936-955399ac0259 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:35.183281 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1191/2397] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:41:35 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:41:35.187453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e54f8c-4cd5-42b7-b56a-d52b15e9e8cd admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:35.198780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e54f8c-4cd5-42b7-b56a-d52b15e9e8cd admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:35.259770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee6424-da67-4020-9e6d-0f690655168c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:35.260450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee6424-da67-4020-9e6d-0f690655168c admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:35.260740 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee6424-da67-4020-9e6d-0f690655168c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:35.261019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee6424-da67-4020-9e6d-0f690655168c admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:35.261712 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee6424-da67-4020-9e6d-0f690655168c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:35.263647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee6424-da67-4020-9e6d-0f690655168c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:35.283381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1207/2398] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:41:35 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:41:35.287553 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50005b-3bb9-4593-a936-955399ac0259 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:35.298095 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50005b-3bb9-4593-a936-955399ac0259 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:35.302795 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8d51d-08fa-4dcf-9500-73bd6c2ee04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:35.303442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8d51d-08fa-4dcf-9500-73bd6c2ee04b admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:35.303718 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8d51d-08fa-4dcf-9500-73bd6c2ee04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:35.304019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8d51d-08fa-4dcf-9500-73bd6c2ee04b admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:35.304796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8d51d-08fa-4dcf-9500-73bd6c2ee04b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:35.307226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8d51d-08fa-4dcf-9500-73bd6c2ee04b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:35.317664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1192/2399] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:41:35.322543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee6424-da67-4020-9e6d-0f690655168c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:35.332575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee6424-da67-4020-9e6d-0f690655168c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:35.337273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b147ce2-bc61-4daf-b402-30e81eee863e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:35.337920 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b147ce2-bc61-4daf-b402-30e81eee863e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:35.338200 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b147ce2-bc61-4daf-b402-30e81eee863e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:35.338470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b147ce2-bc61-4daf-b402-30e81eee863e admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:35.339184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b147ce2-bc61-4daf-b402-30e81eee863e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:35.340806 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b147ce2-bc61-4daf-b402-30e81eee863e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:35.350601 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1208/2400] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:41:35.354775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8d51d-08fa-4dcf-9500-73bd6c2ee04b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:35.364645 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8d51d-08fa-4dcf-9500-73bd6c2ee04b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:35.369654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604612c-21f6-4a31-a076-3f452fde80d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:35.370522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604612c-21f6-4a31-a076-3f452fde80d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:35.370804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604612c-21f6-4a31-a076-3f452fde80d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:35.371079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3604612c-21f6-4a31-a076-3f452fde80d1 admin admin] PATH_INFO: `/v3/projects/96b471c181204a1582fe92816718094c/users/b193b1915b1a42cbaf67e27144e4c060/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:35.386053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604612c-21f6-4a31-a076-3f452fde80d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b471c181204a1582fe92816718094c, user_id=b193b1915b1a42cbaf67e27144e4c060, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:35.388782 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3604612c-21f6-4a31-a076-3f452fde80d1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:35.401441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1193/2401] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:41:35 2022] PUT /identity/v3/projects/96b471c181204a1582fe92816718094c/users/b193b1915b1a42cbaf67e27144e4c060/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:41:35.411952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087600c3-6213-41d8-baf3-6ae1b4991c3d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:35.412255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087600c3-6213-41d8-baf3-6ae1b4991c3d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:35.412539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087600c3-6213-41d8-baf3-6ae1b4991c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:35.440039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-087600c3-6213-41d8-baf3-6ae1b4991c3d None None] MFA Rules not processed for user `b193b1915b1a42cbaf67e27144e4c060`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:41:35.441266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-087600c3-6213-41d8-baf3-6ae1b4991c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:35.505549 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1209/2402] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:41:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:41:35.517462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604612c-21f6-4a31-a076-3f452fde80d1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:41:35.519137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3604612c-21f6-4a31-a076-3f452fde80d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:41:35.549180 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:35.599851 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3604612c-21f6-4a31-a076-3f452fde80d1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:41:35.603874 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ff29d-0ea6-4731-9f90-e1476632e992 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e053b6b25a4e4a20a05e6ffa6683222b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:41:35.604570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ff29d-0ea6-4731-9f90-e1476632e992 service glance] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:41:35.604857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ff29d-0ea6-4731-9f90-e1476632e992 service glance] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:41:35.605137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ff29d-0ea6-4731-9f90-e1476632e992 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:41:35.613519 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ff29d-0ea6-4731-9f90-e1476632e992 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:41:35.615021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ff29d-0ea6-4731-9f90-e1476632e992 service glance] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:41:35.623845 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1194/2403] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:41:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:41:45.502556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:41:45.619784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=23 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:43:17.030163 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087600c3-6213-41d8-baf3-6ae1b4991c3d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:43:17.031622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-087600c3-6213-41d8-baf3-6ae1b4991c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:43:17.110136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087600c3-6213-41d8-baf3-6ae1b4991c3d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:43:17.115345 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f49434-e183-4ba4-a5ea-7d08f6bafdc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:43:17.116179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f49434-e183-4ba4-a5ea-7d08f6bafdc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:43:17.116470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f49434-e183-4ba4-a5ea-7d08f6bafdc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:43:17.116752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f49434-e183-4ba4-a5ea-7d08f6bafdc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:43:17.118572 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-52f49434-e183-4ba4-a5ea-7d08f6bafdc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:43:17.129605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f49434-e183-4ba4-a5ea-7d08f6bafdc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:43:17.131059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f49434-e183-4ba4-a5ea-7d08f6bafdc7 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:43:17.140121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1210/2404] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:43:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:43:27.136202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=26 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:44:09.291423 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ff29d-0ea6-4731-9f90-e1476632e992 service glance] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:44:09.292889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8e7ff29d-0ea6-4731-9f90-e1476632e992 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:44:09.363942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ff29d-0ea6-4731-9f90-e1476632e992 service glance] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:44:09.369081 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf786e-1d38-4db6-a86f-9d8ac4627f25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:44:09.369731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf786e-1d38-4db6-a86f-9d8ac4627f25 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:44:09.370017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf786e-1d38-4db6-a86f-9d8ac4627f25 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:44:09.370299 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf786e-1d38-4db6-a86f-9d8ac4627f25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:44:09.372123 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-25bf786e-1d38-4db6-a86f-9d8ac4627f25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:44:09.384575 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf786e-1d38-4db6-a86f-9d8ac4627f25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:44:09.386185 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf786e-1d38-4db6-a86f-9d8ac4627f25 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:44:09.396629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1195/2405] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:44:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:44:19.392051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=32 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:44:24.117302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f49434-e183-4ba4-a5ea-7d08f6bafdc7 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:44:24.118725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-52f49434-e183-4ba4-a5ea-7d08f6bafdc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:44:24.176956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f49434-e183-4ba4-a5ea-7d08f6bafdc7 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:44:24.180606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cf446-bdd7-4d46-a219-e10a54a204d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:44:24.181261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cf446-bdd7-4d46-a219-e10a54a204d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:44:24.181541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cf446-bdd7-4d46-a219-e10a54a204d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:44:24.181816 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cf446-bdd7-4d46-a219-e10a54a204d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:44:24.183621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-df9cf446-bdd7-4d46-a219-e10a54a204d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:44:24.258025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cf446-bdd7-4d46-a219-e10a54a204d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:44:24.259481 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cf446-bdd7-4d46-a219-e10a54a204d9 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:44:24.280085 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1211/2406] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:44:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:44:34.276760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=56 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:45:24.425544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf786e-1d38-4db6-a86f-9d8ac4627f25 service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:24.427192 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-25bf786e-1d38-4db6-a86f-9d8ac4627f25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:24.442057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf786e-1d38-4db6-a86f-9d8ac4627f25 service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:24.446910 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973722c5-ff4d-418b-a5ea-c3f21bb8dd80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:24.448804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973722c5-ff4d-418b-a5ea-c3f21bb8dd80 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:24.448804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973722c5-ff4d-418b-a5ea-c3f21bb8dd80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:24.449239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973722c5-ff4d-418b-a5ea-c3f21bb8dd80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:24.451799 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-973722c5-ff4d-418b-a5ea-c3f21bb8dd80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:24.533400 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973722c5-ff4d-418b-a5ea-c3f21bb8dd80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:24.535852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973722c5-ff4d-418b-a5ea-c3f21bb8dd80 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:24.553839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1196/2407] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:45:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:45:26.460155 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cf446-bdd7-4d46-a219-e10a54a204d9 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:26.463359 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-df9cf446-bdd7-4d46-a219-e10a54a204d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:26.601761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cf446-bdd7-4d46-a219-e10a54a204d9 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:26.612024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57563d-c7d4-424d-9d1f-48a82b312fbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:26.613391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57563d-c7d4-424d-9d1f-48a82b312fbc service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:26.613903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57563d-c7d4-424d-9d1f-48a82b312fbc service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:26.614181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57563d-c7d4-424d-9d1f-48a82b312fbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:26.617250 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-eb57563d-c7d4-424d-9d1f-48a82b312fbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:26.652680 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57563d-c7d4-424d-9d1f-48a82b312fbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:26.659022 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57563d-c7d4-424d-9d1f-48a82b312fbc service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:26.695019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1212/2408] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:45:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:45:28.746375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973722c5-ff4d-418b-a5ea-c3f21bb8dd80 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:28.749178 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-973722c5-ff4d-418b-a5ea-c3f21bb8dd80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:28.767753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973722c5-ff4d-418b-a5ea-c3f21bb8dd80 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:28.773160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546be14-c025-4407-8b54-6e8db450ab5e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:28.774749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546be14-c025-4407-8b54-6e8db450ab5e service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:28.775147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546be14-c025-4407-8b54-6e8db450ab5e service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:28.775797 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546be14-c025-4407-8b54-6e8db450ab5e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:28.778000 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3546be14-c025-4407-8b54-6e8db450ab5e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:28.792399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546be14-c025-4407-8b54-6e8db450ab5e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:28.796266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546be14-c025-4407-8b54-6e8db450ab5e service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:28.813775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1197/2409] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:45:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:45:28.830502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57563d-c7d4-424d-9d1f-48a82b312fbc service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:28.832593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-eb57563d-c7d4-424d-9d1f-48a82b312fbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:28.854331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57563d-c7d4-424d-9d1f-48a82b312fbc service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:28.861846 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d291fa-132c-4091-b980-07ae8dd101b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:28.862763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d291fa-132c-4091-b980-07ae8dd101b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:28.863046 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d291fa-132c-4091-b980-07ae8dd101b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:28.863321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d291fa-132c-4091-b980-07ae8dd101b8 admin admin] PATH_INFO: `/v3/users/6c62db4b38434cfe886108a72e4c42aa` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:28.865528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d291fa-132c-4091-b980-07ae8dd101b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c62db4b38434cfe886108a72e4c42aa)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:28.867460 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d291fa-132c-4091-b980-07ae8dd101b8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:28.949073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b5d291fa-132c-4091-b980-07ae8dd101b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c62db4b38434cfe886108a72e4c42aa'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:45:28.949846 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b5d291fa-132c-4091-b980-07ae8dd101b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c62db4b38434cfe886108a72e4c42aa'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:45:28.959760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b5d291fa-132c-4091-b980-07ae8dd101b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c62db4b38434cfe886108a72e4c42aa'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:45:28.977139 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b5d291fa-132c-4091-b980-07ae8dd101b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c62db4b38434cfe886108a72e4c42aa'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:45:28.987289 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b5d291fa-132c-4091-b980-07ae8dd101b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c62db4b38434cfe886108a72e4c42aa'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:45:28.994322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1213/2410] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:45:28 2022] DELETE /identity/v3/users/6c62db4b38434cfe886108a72e4c42aa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:45:29.216615 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546be14-c025-4407-8b54-6e8db450ab5e service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:29.233115 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546be14-c025-4407-8b54-6e8db450ab5e service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:29.307597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb086d-6d65-4eea-ae35-121ac8927acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:29.308544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb086d-6d65-4eea-ae35-121ac8927acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:29.308975 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb086d-6d65-4eea-ae35-121ac8927acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:29.309265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb086d-6d65-4eea-ae35-121ac8927acf admin admin] PATH_INFO: `/v3/projects/b47c6d295e05423a9e5ad6f0499cbf2a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:29.312105 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb086d-6d65-4eea-ae35-121ac8927acf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b47c6d295e05423a9e5ad6f0499cbf2a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:29.314107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb086d-6d65-4eea-ae35-121ac8927acf admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:29.364453 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ddb086d-6d65-4eea-ae35-121ac8927acf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b47c6d295e05423a9e5ad6f0499cbf2a', 'request_id': 'req-0ddb086d-6d65-4eea-ae35-121ac8927acf'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:45:29.370947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ddb086d-6d65-4eea-ae35-121ac8927acf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b47c6d295e05423a9e5ad6f0499cbf2a', 'request_id': 'req-0ddb086d-6d65-4eea-ae35-121ac8927acf'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:45:29.380304 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ddb086d-6d65-4eea-ae35-121ac8927acf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b47c6d295e05423a9e5ad6f0499cbf2a', 'request_id': 'req-0ddb086d-6d65-4eea-ae35-121ac8927acf'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:45:29.391845 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ddb086d-6d65-4eea-ae35-121ac8927acf admin admin] The token cache is being invalidate because project b47c6d295e05423a9e5ad6f0499cbf2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:45:29.392163 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ddb086d-6d65-4eea-ae35-121ac8927acf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:45:29.392878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ddb086d-6d65-4eea-ae35-121ac8927acf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:45:29.394673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1198/2411] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:45:29 2022] DELETE /identity/v3/projects/b47c6d295e05423a9e5ad6f0499cbf2a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:45:29.401671 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411debd-e680-48ea-8d0c-9f52566dfbb4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:29.402118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411debd-e680-48ea-8d0c-9f52566dfbb4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:29.402478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411debd-e680-48ea-8d0c-9f52566dfbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:29.421143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e411debd-e680-48ea-8d0c-9f52566dfbb4 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:45:29.422351 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e411debd-e680-48ea-8d0c-9f52566dfbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:29.467005 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:45:29.493786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1214/2412] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:45:29.495567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351cadc-64d7-4433-814c-f1a5c3002a60 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:29.495957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351cadc-64d7-4433-814c-f1a5c3002a60 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:29.496316 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351cadc-64d7-4433-814c-f1a5c3002a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:29.518197 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6351cadc-64d7-4433-814c-f1a5c3002a60 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:45:29.519465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6351cadc-64d7-4433-814c-f1a5c3002a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:29.527629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1199/2413] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:45:29.532588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411debd-e680-48ea-8d0c-9f52566dfbb4 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:29.542496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411debd-e680-48ea-8d0c-9f52566dfbb4 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:29.548821 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d6c7c-603e-451a-ba0b-324d5fe2e75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:29.549546 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d6c7c-603e-451a-ba0b-324d5fe2e75c admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:29.549957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d6c7c-603e-451a-ba0b-324d5fe2e75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:29.550263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d6c7c-603e-451a-ba0b-324d5fe2e75c admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:29.551037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d6c7c-603e-451a-ba0b-324d5fe2e75c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:45:29.551347 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d6c7c-603e-451a-ba0b-324d5fe2e75c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:29.553408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d6c7c-603e-451a-ba0b-324d5fe2e75c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:29.566571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1215/2414] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:45:29.571044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351cadc-64d7-4433-814c-f1a5c3002a60 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:29.577012 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:45:29.581659 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351cadc-64d7-4433-814c-f1a5c3002a60 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:29.588274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ef067-8f88-4d7c-bae4-ccbe8cd1cdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:29.589079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ef067-8f88-4d7c-bae4-ccbe8cd1cdf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:29.589712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ef067-8f88-4d7c-bae4-ccbe8cd1cdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:29.590602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ef067-8f88-4d7c-bae4-ccbe8cd1cdf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:29.592048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ef067-8f88-4d7c-bae4-ccbe8cd1cdf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:29.594156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ef067-8f88-4d7c-bae4-ccbe8cd1cdf8 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:29.624548 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1200/2415] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:45:29 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:45:29.631392 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d6c7c-603e-451a-ba0b-324d5fe2e75c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:29.643381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d6c7c-603e-451a-ba0b-324d5fe2e75c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:29.691609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907370a-4c24-4c56-8a53-001403fd18da None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:29.691876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907370a-4c24-4c56-8a53-001403fd18da None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:29.692202 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907370a-4c24-4c56-8a53-001403fd18da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:29.710551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ba787-bbc4-407f-8e84-0491ebb4f328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:29.711360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ba787-bbc4-407f-8e84-0491ebb4f328 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:29.711751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ba787-bbc4-407f-8e84-0491ebb4f328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:29.712162 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ba787-bbc4-407f-8e84-0491ebb4f328 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:29.713011 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ba787-bbc4-407f-8e84-0491ebb4f328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:29.715000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ba787-bbc4-407f-8e84-0491ebb4f328 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:29.726211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b907370a-4c24-4c56-8a53-001403fd18da None None] MFA Rules not processed for user `7b4006c43f7944c083cf323db4df2b3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:45:29.727471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b907370a-4c24-4c56-8a53-001403fd18da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:29.735466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1216/2416] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:45:29 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:45:29.739982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ba787-bbc4-407f-8e84-0491ebb4f328 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:29.750998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ba787-bbc4-407f-8e84-0491ebb4f328 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:29.757307 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3718c1-028a-46c1-8033-0df7b74c871f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:29.758042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3718c1-028a-46c1-8033-0df7b74c871f admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:29.758336 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3718c1-028a-46c1-8033-0df7b74c871f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:29.758606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3718c1-028a-46c1-8033-0df7b74c871f admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:29.759600 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3718c1-028a-46c1-8033-0df7b74c871f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:29.761415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3718c1-028a-46c1-8033-0df7b74c871f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:29.773309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1217/2417] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:45:29.776842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3718c1-028a-46c1-8033-0df7b74c871f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:29.787165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3718c1-028a-46c1-8033-0df7b74c871f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:29.792275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a8e20-ad7b-4db9-a7d1-1e9ec2b04b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:29.793000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a8e20-ad7b-4db9-a7d1-1e9ec2b04b45 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:29.793359 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a8e20-ad7b-4db9-a7d1-1e9ec2b04b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:29.793710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a8e20-ad7b-4db9-a7d1-1e9ec2b04b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:29.795114 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a8e20-ad7b-4db9-a7d1-1e9ec2b04b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:29.796838 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a8e20-ad7b-4db9-a7d1-1e9ec2b04b45 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:29.803176 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1201/2418] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:45:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:45:29.811968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1218/2419] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:45:29.815967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907370a-4c24-4c56-8a53-001403fd18da None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:29.828366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907370a-4c24-4c56-8a53-001403fd18da None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:29.832959 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7a867-8439-4624-89b1-c76db1f1fa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:29.833840 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7a867-8439-4624-89b1-c76db1f1fa8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:29.834120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7a867-8439-4624-89b1-c76db1f1fa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:29.834394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7a867-8439-4624-89b1-c76db1f1fa8c admin admin] PATH_INFO: `/v3/projects/dae2be7fd0bc46fcabdcba37d239a486/users/bfe5e16147c74ecc96be44cf04642eec/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:29.855089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7a867-8439-4624-89b1-c76db1f1fa8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae2be7fd0bc46fcabdcba37d239a486, user_id=bfe5e16147c74ecc96be44cf04642eec, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:29.858191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7a867-8439-4624-89b1-c76db1f1fa8c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:29.876028 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1202/2420] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:45:29 2022] PUT /identity/v3/projects/dae2be7fd0bc46fcabdcba37d239a486/users/bfe5e16147c74ecc96be44cf04642eec/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:45:29.893378 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ac32e-7985-4970-8b4c-0ec69fa3fc9b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:29.894005 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ac32e-7985-4970-8b4c-0ec69fa3fc9b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:29.894380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ac32e-7985-4970-8b4c-0ec69fa3fc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:29.929592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-934ac32e-7985-4970-8b4c-0ec69fa3fc9b None None] MFA Rules not processed for user `bfe5e16147c74ecc96be44cf04642eec`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:45:29.930969 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-934ac32e-7985-4970-8b4c-0ec69fa3fc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:30.012701 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1219/2421] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:45:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:45:30.017818 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7a867-8439-4624-89b1-c76db1f1fa8c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:30.081957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7a867-8439-4624-89b1-c76db1f1fa8c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:30.089182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4330e-ddef-48a3-ad80-7988df29238f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:30.089182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4330e-ddef-48a3-ad80-7988df29238f admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:30.089461 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4330e-ddef-48a3-ad80-7988df29238f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:30.089691 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4330e-ddef-48a3-ad80-7988df29238f admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:30.090365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4330e-ddef-48a3-ad80-7988df29238f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:30.092990 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4330e-ddef-48a3-ad80-7988df29238f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:30.126609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1203/2422] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:45:30 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:45:30.130627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ac32e-7985-4970-8b4c-0ec69fa3fc9b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:30.141782 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ac32e-7985-4970-8b4c-0ec69fa3fc9b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:30.165049 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4330e-ddef-48a3-ad80-7988df29238f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:30.166945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-31a4330e-ddef-48a3-ad80-7988df29238f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:30.173123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:45:30.203591 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:45:30.203870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98412179-36ee-4067-a879-d0fc714a9bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:30.204539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98412179-36ee-4067-a879-d0fc714a9bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:30.204664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98412179-36ee-4067-a879-d0fc714a9bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:30.204917 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98412179-36ee-4067-a879-d0fc714a9bc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:30.205621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98412179-36ee-4067-a879-d0fc714a9bc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:30.207580 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98412179-36ee-4067-a879-d0fc714a9bc6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:30.229306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1220/2423] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:45:30 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:45:30.233313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98412179-36ee-4067-a879-d0fc714a9bc6 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:30.234965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4330e-ddef-48a3-ad80-7988df29238f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:30.239291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c961081-d261-4d01-be60-9f581f7ec296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:30.239969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c961081-d261-4d01-be60-9f581f7ec296 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:30.240256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c961081-d261-4d01-be60-9f581f7ec296 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:30.240535 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c961081-d261-4d01-be60-9f581f7ec296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:30.242308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1c961081-d261-4d01-be60-9f581f7ec296 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:30.243225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98412179-36ee-4067-a879-d0fc714a9bc6 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:30.248445 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be4f3b-0ad9-4100-9304-c48cb949fad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:30.249101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be4f3b-0ad9-4100-9304-c48cb949fad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:30.249389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be4f3b-0ad9-4100-9304-c48cb949fad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:30.249700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be4f3b-0ad9-4100-9304-c48cb949fad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:30.250427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be4f3b-0ad9-4100-9304-c48cb949fad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:30.252102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be4f3b-0ad9-4100-9304-c48cb949fad8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:30.252706 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c961081-d261-4d01-be60-9f581f7ec296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:30.254182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c961081-d261-4d01-be60-9f581f7ec296 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:30.262175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1221/2424] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:45:30.262353 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1204/2425] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:45:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:45:30.267157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be4f3b-0ad9-4100-9304-c48cb949fad8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:30.277109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be4f3b-0ad9-4100-9304-c48cb949fad8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:30.283155 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10c0b1-25f1-4f2e-9c5e-91816c974eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:30.284103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10c0b1-25f1-4f2e-9c5e-91816c974eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:30.284391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10c0b1-25f1-4f2e-9c5e-91816c974eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:30.284682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10c0b1-25f1-4f2e-9c5e-91816c974eeb admin admin] PATH_INFO: `/v3/projects/5b7de874b82d4de0a6fe20737c9f99c2/users/3063408e632d405c955f7d233573bf73/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:30.300315 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10c0b1-25f1-4f2e-9c5e-91816c974eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7de874b82d4de0a6fe20737c9f99c2, user_id=3063408e632d405c955f7d233573bf73, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:30.303005 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10c0b1-25f1-4f2e-9c5e-91816c974eeb admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:30.317641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1222/2426] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:45:30 2022] PUT /identity/v3/projects/5b7de874b82d4de0a6fe20737c9f99c2/users/3063408e632d405c955f7d233573bf73/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:45:30.321683 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c961081-d261-4d01-be60-9f581f7ec296 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:30.334148 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c961081-d261-4d01-be60-9f581f7ec296 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:30.395036 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800942f-f86b-4faa-9f39-3cd3ec8cf46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:30.395712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800942f-f86b-4faa-9f39-3cd3ec8cf46d admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:30.396027 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800942f-f86b-4faa-9f39-3cd3ec8cf46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:30.396308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800942f-f86b-4faa-9f39-3cd3ec8cf46d admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:30.397035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800942f-f86b-4faa-9f39-3cd3ec8cf46d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:30.398664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800942f-f86b-4faa-9f39-3cd3ec8cf46d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:30.409766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1205/2427] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:45:30.414172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10c0b1-25f1-4f2e-9c5e-91816c974eeb admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:30.424921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10c0b1-25f1-4f2e-9c5e-91816c974eeb admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:30.429511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5162c-ea9d-499f-b5f3-98021642fbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:30.430399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5162c-ea9d-499f-b5f3-98021642fbc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:30.430681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5162c-ea9d-499f-b5f3-98021642fbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:30.430959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5162c-ea9d-499f-b5f3-98021642fbc0 admin admin] PATH_INFO: `/v3/projects/5b7de874b82d4de0a6fe20737c9f99c2/users/3063408e632d405c955f7d233573bf73/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:30.432967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5162c-ea9d-499f-b5f3-98021642fbc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7de874b82d4de0a6fe20737c9f99c2, user_id=3063408e632d405c955f7d233573bf73, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:30.436198 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5162c-ea9d-499f-b5f3-98021642fbc0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:30.446801 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1223/2428] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:45:30 2022] PUT /identity/v3/projects/5b7de874b82d4de0a6fe20737c9f99c2/users/3063408e632d405c955f7d233573bf73/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:45:30.458207 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfd400-a4fa-4093-a51a-37bdd42be496 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:30.458512 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfd400-a4fa-4093-a51a-37bdd42be496 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:30.458784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfd400-a4fa-4093-a51a-37bdd42be496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:30.490742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0dbfd400-a4fa-4093-a51a-37bdd42be496 None None] MFA Rules not processed for user `3063408e632d405c955f7d233573bf73`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:45:30.491989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0dbfd400-a4fa-4093-a51a-37bdd42be496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:30.567676 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1206/2429] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:45:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:45:30.580981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5162c-ea9d-499f-b5f3-98021642fbc0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:30.582768 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b3b5162c-ea9d-499f-b5f3-98021642fbc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:30.621780 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfd400-a4fa-4093-a51a-37bdd42be496 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:30.623445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0dbfd400-a4fa-4093-a51a-37bdd42be496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:30.656753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5162c-ea9d-499f-b5f3-98021642fbc0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:30.663675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4872d-8f49-4b69-8cb8-816afed6cf4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:30.664443 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4872d-8f49-4b69-8cb8-816afed6cf4f service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:30.664796 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4872d-8f49-4b69-8cb8-816afed6cf4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:30.665141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4872d-8f49-4b69-8cb8-816afed6cf4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:30.677188 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4872d-8f49-4b69-8cb8-816afed6cf4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:30.678769 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4872d-8f49-4b69-8cb8-816afed6cf4f service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:30.689788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1224/2430] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:45:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:45:30.693696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:45:30.711102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfd400-a4fa-4093-a51a-37bdd42be496 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:30.719023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777c134-97ae-4067-90d2-ebfa33a533fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:30.719816 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777c134-97ae-4067-90d2-ebfa33a533fb service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:30.720132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777c134-97ae-4067-90d2-ebfa33a533fb service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:30.720463 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777c134-97ae-4067-90d2-ebfa33a533fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:30.722708 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6777c134-97ae-4067-90d2-ebfa33a533fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:30.735971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777c134-97ae-4067-90d2-ebfa33a533fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:30.737614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777c134-97ae-4067-90d2-ebfa33a533fb service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:30.749196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1207/2431] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:45:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:45:30.771350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4872d-8f49-4b69-8cb8-816afed6cf4f service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:30.780254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4872d-8f49-4b69-8cb8-816afed6cf4f service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:30.784094 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08801297-6072-40aa-9aef-eddf8edecc9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:30.784860 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08801297-6072-40aa-9aef-eddf8edecc9e service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:30.785136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08801297-6072-40aa-9aef-eddf8edecc9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:30.785397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08801297-6072-40aa-9aef-eddf8edecc9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:30.846179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08801297-6072-40aa-9aef-eddf8edecc9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:30.847678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08801297-6072-40aa-9aef-eddf8edecc9e service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:30.857545 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1225/2432] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:45:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:45:38.299604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777c134-97ae-4067-90d2-ebfa33a533fb service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:38.301545 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6777c134-97ae-4067-90d2-ebfa33a533fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:38.390170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777c134-97ae-4067-90d2-ebfa33a533fb service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:38.395930 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e654a60-6352-47d9-b9ef-ffb784112bf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:38.396539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e654a60-6352-47d9-b9ef-ffb784112bf7 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:38.396818 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e654a60-6352-47d9-b9ef-ffb784112bf7 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:38.397092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e654a60-6352-47d9-b9ef-ffb784112bf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:38.398936 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3e654a60-6352-47d9-b9ef-ffb784112bf7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:38.411935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e654a60-6352-47d9-b9ef-ffb784112bf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:38.413560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e654a60-6352-47d9-b9ef-ffb784112bf7 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:38.424896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1208/2433] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:45:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:45:40.853186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:45:42.260041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08801297-6072-40aa-9aef-eddf8edecc9e service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:45:42.261734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-08801297-6072-40aa-9aef-eddf8edecc9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:42.279127 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08801297-6072-40aa-9aef-eddf8edecc9e service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:45:42.285396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b074c9-9c27-4463-bb73-98f25b692f20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:45:42.285623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b074c9-9c27-4463-bb73-98f25b692f20 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:45:42.285728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b074c9-9c27-4463-bb73-98f25b692f20 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:45:42.286007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b074c9-9c27-4463-bb73-98f25b692f20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:45:42.287809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b1b074c9-9c27-4463-bb73-98f25b692f20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:45:42.300988 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b074c9-9c27-4463-bb73-98f25b692f20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:45:42.302590 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b074c9-9c27-4463-bb73-98f25b692f20 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:45:42.313929 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1226/2434] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:45:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:45:48.420255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=41 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:45:52.308830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:24.211484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e654a60-6352-47d9-b9ef-ffb784112bf7 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:24.221639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e654a60-6352-47d9-b9ef-ffb784112bf7 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:24.225467 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c10222-9bcd-4dd4-9dfa-034b0bd3cfb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:24.226116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c10222-9bcd-4dd4-9dfa-034b0bd3cfb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:24.226404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c10222-9bcd-4dd4-9dfa-034b0bd3cfb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:24.226683 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c10222-9bcd-4dd4-9dfa-034b0bd3cfb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:24.228570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-17c10222-9bcd-4dd4-9dfa-034b0bd3cfb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:24.300028 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c10222-9bcd-4dd4-9dfa-034b0bd3cfb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:24.301488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c10222-9bcd-4dd4-9dfa-034b0bd3cfb9 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:24.321859 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1209/2435] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:26.371045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b074c9-9c27-4463-bb73-98f25b692f20 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:26.372488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b1b074c9-9c27-4463-bb73-98f25b692f20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:26.386016 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b074c9-9c27-4463-bb73-98f25b692f20 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:26.389840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27c5b7-1275-4ac6-99cb-21d6c1ae1806 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:26.390483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27c5b7-1275-4ac6-99cb-21d6c1ae1806 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:26.390754 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27c5b7-1275-4ac6-99cb-21d6c1ae1806 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:26.391032 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27c5b7-1275-4ac6-99cb-21d6c1ae1806 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:26.392922 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0b27c5b7-1275-4ac6-99cb-21d6c1ae1806 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:26.460973 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27c5b7-1275-4ac6-99cb-21d6c1ae1806 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:26.462422 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27c5b7-1275-4ac6-99cb-21d6c1ae1806 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:26.472852 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1227/2436] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:34.318481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=36 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:34.645385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c10222-9bcd-4dd4-9dfa-034b0bd3cfb9 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:34.650603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c10222-9bcd-4dd4-9dfa-034b0bd3cfb9 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:34.654510 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99424946-8180-4552-9164-f82f0151362a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:34.655168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99424946-8180-4552-9164-f82f0151362a service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:34.655451 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99424946-8180-4552-9164-f82f0151362a service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:34.655745 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99424946-8180-4552-9164-f82f0151362a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:34.657586 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-99424946-8180-4552-9164-f82f0151362a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:34.718639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99424946-8180-4552-9164-f82f0151362a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:34.720164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99424946-8180-4552-9164-f82f0151362a service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:34.729571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1210/2437] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:36.468113 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=28 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:40.555698 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27c5b7-1275-4ac6-99cb-21d6c1ae1806 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:40.557489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0b27c5b7-1275-4ac6-99cb-21d6c1ae1806 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:40.630030 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27c5b7-1275-4ac6-99cb-21d6c1ae1806 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:40.634818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1c661-21fa-424b-92cb-ce8da1365bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:40.635697 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1c661-21fa-424b-92cb-ce8da1365bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:40.636007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1c661-21fa-424b-92cb-ce8da1365bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:40.636290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1c661-21fa-424b-92cb-ce8da1365bdf admin admin] PATH_INFO: `/v3/users/c88c3fce08f64bc3aff957ed842054e5` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:40.637472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1c661-21fa-424b-92cb-ce8da1365bdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c88c3fce08f64bc3aff957ed842054e5)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:40.639694 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1c661-21fa-424b-92cb-ce8da1365bdf admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:40.689312 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-44e1c661-21fa-424b-92cb-ce8da1365bdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c88c3fce08f64bc3aff957ed842054e5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:40.689957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-44e1c661-21fa-424b-92cb-ce8da1365bdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c88c3fce08f64bc3aff957ed842054e5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:40.697913 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-44e1c661-21fa-424b-92cb-ce8da1365bdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c88c3fce08f64bc3aff957ed842054e5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:40.711681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-44e1c661-21fa-424b-92cb-ce8da1365bdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c88c3fce08f64bc3aff957ed842054e5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:40.724321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-44e1c661-21fa-424b-92cb-ce8da1365bdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c88c3fce08f64bc3aff957ed842054e5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:40.733777 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1228/2438] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:46:40 2022] DELETE /identity/v3/users/c88c3fce08f64bc3aff957ed842054e5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:40.738522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbbcac-0e51-4592-ae87-623393fe9475 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:40.739033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbbcac-0e51-4592-ae87-623393fe9475 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:40.739155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbbcac-0e51-4592-ae87-623393fe9475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:40.767566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-97cbbcac-0e51-4592-ae87-623393fe9475 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:40.768879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-97cbbcac-0e51-4592-ae87-623393fe9475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:40.836241 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1211/2439] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:40.847037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1c661-21fa-424b-92cb-ce8da1365bdf admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:40.848494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-44e1c661-21fa-424b-92cb-ce8da1365bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:40.922991 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1c661-21fa-424b-92cb-ce8da1365bdf admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:40.927298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1c9f1-1a5a-4223-bb29-35454d927e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:40.928064 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1c9f1-1a5a-4223-bb29-35454d927e7b service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:40.928352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1c9f1-1a5a-4223-bb29-35454d927e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:40.928632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1c9f1-1a5a-4223-bb29-35454d927e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:40.938989 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1c9f1-1a5a-4223-bb29-35454d927e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:40.940472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1c9f1-1a5a-4223-bb29-35454d927e7b service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:40.951451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1229/2440] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:41.246852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbbcac-0e51-4592-ae87-623393fe9475 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:41.248492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-97cbbcac-0e51-4592-ae87-623393fe9475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:41.263235 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbbcac-0e51-4592-ae87-623393fe9475 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:41.267979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd46662-b945-41e0-8305-9b5419ce5c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:41.269054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd46662-b945-41e0-8305-9b5419ce5c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:41.269339 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd46662-b945-41e0-8305-9b5419ce5c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:41.269613 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd46662-b945-41e0-8305-9b5419ce5c4a admin admin] PATH_INFO: `/v3/projects/1bc0608eb848440ba381d11bb5ef56ea` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:41.270807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd46662-b945-41e0-8305-9b5419ce5c4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc0608eb848440ba381d11bb5ef56ea)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:41.272853 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd46662-b945-41e0-8305-9b5419ce5c4a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:41.326662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fdd46662-b945-41e0-8305-9b5419ce5c4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc0608eb848440ba381d11bb5ef56ea', 'request_id': 'req-fdd46662-b945-41e0-8305-9b5419ce5c4a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:41.332770 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fdd46662-b945-41e0-8305-9b5419ce5c4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc0608eb848440ba381d11bb5ef56ea', 'request_id': 'req-fdd46662-b945-41e0-8305-9b5419ce5c4a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:41.340245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fdd46662-b945-41e0-8305-9b5419ce5c4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc0608eb848440ba381d11bb5ef56ea', 'request_id': 'req-fdd46662-b945-41e0-8305-9b5419ce5c4a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:41.352270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fdd46662-b945-41e0-8305-9b5419ce5c4a admin admin] The token cache is being invalidate because project 1bc0608eb848440ba381d11bb5ef56ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:46:41.352495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fdd46662-b945-41e0-8305-9b5419ce5c4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:41.353184 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fdd46662-b945-41e0-8305-9b5419ce5c4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:41.354744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1212/2441] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:46:41 2022] DELETE /identity/v3/projects/1bc0608eb848440ba381d11bb5ef56ea => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:41.360405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463074e-b076-4699-aaf1-f72851c4739a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:41.360695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463074e-b076-4699-aaf1-f72851c4739a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:41.360971 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463074e-b076-4699-aaf1-f72851c4739a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:41.375182 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:41.411799 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f463074e-b076-4699-aaf1-f72851c4739a None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:41.413011 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f463074e-b076-4699-aaf1-f72851c4739a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:41.482383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1230/2442] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:41.490293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74e566-dd20-4489-bf7c-27fac69f077d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:41.490994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74e566-dd20-4489-bf7c-27fac69f077d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:41.491454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74e566-dd20-4489-bf7c-27fac69f077d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:41.521024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-cd74e566-dd20-4489-bf7c-27fac69f077d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:41.522248 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-cd74e566-dd20-4489-bf7c-27fac69f077d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:41.537721 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1213/2443] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:41.540472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463074e-b076-4699-aaf1-f72851c4739a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:41.551051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463074e-b076-4699-aaf1-f72851c4739a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:41.556992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730800f1-d8b0-4aed-9fca-ef52366845ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:41.557673 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730800f1-d8b0-4aed-9fca-ef52366845ae admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:41.558186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730800f1-d8b0-4aed-9fca-ef52366845ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:41.562160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730800f1-d8b0-4aed-9fca-ef52366845ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:41.562160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730800f1-d8b0-4aed-9fca-ef52366845ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:46:41.562160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730800f1-d8b0-4aed-9fca-ef52366845ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:41.562160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730800f1-d8b0-4aed-9fca-ef52366845ae admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:41.574573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1231/2444] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:46:41.579965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74e566-dd20-4489-bf7c-27fac69f077d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:41.590412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74e566-dd20-4489-bf7c-27fac69f077d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:41.601779 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac724b64-d188-44be-87db-a48093bf94fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:41.602392 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac724b64-d188-44be-87db-a48093bf94fd admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:41.602626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac724b64-d188-44be-87db-a48093bf94fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:41.602926 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac724b64-d188-44be-87db-a48093bf94fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:41.603597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac724b64-d188-44be-87db-a48093bf94fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:41.605557 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac724b64-d188-44be-87db-a48093bf94fd admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:41.610149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:41.645452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1214/2445] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:46:41 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:41.650008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730800f1-d8b0-4aed-9fca-ef52366845ae admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:41.662865 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730800f1-d8b0-4aed-9fca-ef52366845ae admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:41.681729 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac724b64-d188-44be-87db-a48093bf94fd admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:41.683419 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ac724b64-d188-44be-87db-a48093bf94fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:41.734331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2af335-9dbc-441f-a5e3-de290492bea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:41.735147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2af335-9dbc-441f-a5e3-de290492bea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:41.735276 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2af335-9dbc-441f-a5e3-de290492bea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:41.735588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2af335-9dbc-441f-a5e3-de290492bea8 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:41.736370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2af335-9dbc-441f-a5e3-de290492bea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:41.738371 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2af335-9dbc-441f-a5e3-de290492bea8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:41.759032 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1232/2446] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:46:41 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:41.763795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2af335-9dbc-441f-a5e3-de290492bea8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:41.768136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac724b64-d188-44be-87db-a48093bf94fd admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:41.773106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4898d-485a-42fb-91ec-2ef03efb40d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:41.773973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4898d-485a-42fb-91ec-2ef03efb40d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:41.774248 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4898d-485a-42fb-91ec-2ef03efb40d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:41.774529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4898d-485a-42fb-91ec-2ef03efb40d5 admin admin] PATH_INFO: `/v3/users/bfe5e16147c74ecc96be44cf04642eec` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:41.774650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2af335-9dbc-441f-a5e3-de290492bea8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:41.775707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4898d-485a-42fb-91ec-2ef03efb40d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfe5e16147c74ecc96be44cf04642eec)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:41.777701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4898d-485a-42fb-91ec-2ef03efb40d5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:41.780270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b19f6e-fbd6-4805-bd98-937a5eb22f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:41.780962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b19f6e-fbd6-4805-bd98-937a5eb22f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:41.781252 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b19f6e-fbd6-4805-bd98-937a5eb22f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:41.781531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b19f6e-fbd6-4805-bd98-937a5eb22f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:41.782267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b19f6e-fbd6-4805-bd98-937a5eb22f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:41.783860 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b19f6e-fbd6-4805-bd98-937a5eb22f00 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:41.797472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1233/2447] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:46:41.798497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b19f6e-fbd6-4805-bd98-937a5eb22f00 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:41.808356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b19f6e-fbd6-4805-bd98-937a5eb22f00 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:41.814169 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cfb3b-2bfe-49ac-b986-96a5136074d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:41.815047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cfb3b-2bfe-49ac-b986-96a5136074d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:41.815323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cfb3b-2bfe-49ac-b986-96a5136074d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:41.815763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cfb3b-2bfe-49ac-b986-96a5136074d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:41.816617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758cfb3b-2bfe-49ac-b986-96a5136074d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:41.818661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758cfb3b-2bfe-49ac-b986-96a5136074d8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:41.830446 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1234/2448] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:41.833697 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-16e4898d-485a-42fb-91ec-2ef03efb40d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfe5e16147c74ecc96be44cf04642eec'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:41.835298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cfb3b-2bfe-49ac-b986-96a5136074d8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:41.843557 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-16e4898d-485a-42fb-91ec-2ef03efb40d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfe5e16147c74ecc96be44cf04642eec'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:41.846206 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cfb3b-2bfe-49ac-b986-96a5136074d8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:41.851048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b135d-b8e4-42e5-9efa-2bcc1b79dfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:41.851457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-16e4898d-485a-42fb-91ec-2ef03efb40d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfe5e16147c74ecc96be44cf04642eec'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:41.851940 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b135d-b8e4-42e5-9efa-2bcc1b79dfdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:41.852241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b135d-b8e4-42e5-9efa-2bcc1b79dfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:41.852616 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b135d-b8e4-42e5-9efa-2bcc1b79dfdf admin admin] PATH_INFO: `/v3/projects/d7473ead72c3435fa1a1f478b01a41d1/users/bd3b26480fc842d18aaa2f91a1b8fc60/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:41.864464 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-16e4898d-485a-42fb-91ec-2ef03efb40d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfe5e16147c74ecc96be44cf04642eec'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:41.865189 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-16e4898d-485a-42fb-91ec-2ef03efb40d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfe5e16147c74ecc96be44cf04642eec'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:41.869644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b135d-b8e4-42e5-9efa-2bcc1b79dfdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7473ead72c3435fa1a1f478b01a41d1, user_id=bd3b26480fc842d18aaa2f91a1b8fc60, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:41.871626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1215/2449] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:46:41 2022] DELETE /identity/v3/users/bfe5e16147c74ecc96be44cf04642eec => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:46:41.873561 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b135d-b8e4-42e5-9efa-2bcc1b79dfdf admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:41.876520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf155a-2245-4393-b00a-c7cc51b9689c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:41.877112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf155a-2245-4393-b00a-c7cc51b9689c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:41.877112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf155a-2245-4393-b00a-c7cc51b9689c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:41.887435 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1235/2450] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:46:41 2022] PUT /identity/v3/projects/d7473ead72c3435fa1a1f478b01a41d1/users/bd3b26480fc842d18aaa2f91a1b8fc60/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:46:41.895064 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1bcf155a-2245-4393-b00a-c7cc51b9689c None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:41.896336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1bcf155a-2245-4393-b00a-c7cc51b9689c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:41.897694 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee4370-1eed-4001-a4d8-5b4d380303d6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:41.897994 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee4370-1eed-4001-a4d8-5b4d380303d6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:41.898280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee4370-1eed-4001-a4d8-5b4d380303d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:41.929142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-21ee4370-1eed-4001-a4d8-5b4d380303d6 None None] MFA Rules not processed for user `bd3b26480fc842d18aaa2f91a1b8fc60`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:41.930298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-21ee4370-1eed-4001-a4d8-5b4d380303d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:41.935714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:41.951002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:41.964123 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1216/2451] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:46:41.975692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf155a-2245-4393-b00a-c7cc51b9689c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:41.977109 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1bcf155a-2245-4393-b00a-c7cc51b9689c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:42.006700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1236/2452] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:46:42.018470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee4370-1eed-4001-a4d8-5b4d380303d6 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:42.020007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-21ee4370-1eed-4001-a4d8-5b4d380303d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:42.035779 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf155a-2245-4393-b00a-c7cc51b9689c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:42.039521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1145a00-580a-41da-b93f-1a9aadf5dbf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:42.040191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1145a00-580a-41da-b93f-1a9aadf5dbf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:42.040480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1145a00-580a-41da-b93f-1a9aadf5dbf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:42.040757 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1145a00-580a-41da-b93f-1a9aadf5dbf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:42.051051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1145a00-580a-41da-b93f-1a9aadf5dbf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:42.052621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1145a00-580a-41da-b93f-1a9aadf5dbf8 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:42.062604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1217/2453] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:46:42.078142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee4370-1eed-4001-a4d8-5b4d380303d6 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:42.081936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4127b08-f7b6-4063-bd03-c49397a84a7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:42.082566 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4127b08-f7b6-4063-bd03-c49397a84a7c service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:42.082850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4127b08-f7b6-4063-bd03-c49397a84a7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:42.083125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4127b08-f7b6-4063-bd03-c49397a84a7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:42.092656 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4127b08-f7b6-4063-bd03-c49397a84a7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:42.094142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4127b08-f7b6-4063-bd03-c49397a84a7c service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:42.104265 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1237/2454] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:46:42.345426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1145a00-580a-41da-b93f-1a9aadf5dbf8 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:42.347068 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d1145a00-580a-41da-b93f-1a9aadf5dbf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:42.367799 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1145a00-580a-41da-b93f-1a9aadf5dbf8 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:42.375094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:42.376131 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:42.376508 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:42.377361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0 admin admin] PATH_INFO: `/v3/projects/dae2be7fd0bc46fcabdcba37d239a486` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:42.378494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae2be7fd0bc46fcabdcba37d239a486)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:42.380664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:42.445717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae2be7fd0bc46fcabdcba37d239a486', 'request_id': 'req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:42.453640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae2be7fd0bc46fcabdcba37d239a486', 'request_id': 'req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:42.461803 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae2be7fd0bc46fcabdcba37d239a486', 'request_id': 'req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:42.473376 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0 admin admin] The token cache is being invalidate because project dae2be7fd0bc46fcabdcba37d239a486 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:46:42.473663 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:42.474298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:42.475955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1218/2455] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:46:42 2022] DELETE /identity/v3/projects/dae2be7fd0bc46fcabdcba37d239a486 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:42.480056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4127b08-f7b6-4063-bd03-c49397a84a7c service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:42.491813 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4127b08-f7b6-4063-bd03-c49397a84a7c service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:42.550729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393c84e-1d8d-454b-8517-542c72db54a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:42.551601 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393c84e-1d8d-454b-8517-542c72db54a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:42.551942 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393c84e-1d8d-454b-8517-542c72db54a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:42.552174 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393c84e-1d8d-454b-8517-542c72db54a6 admin admin] PATH_INFO: `/v3/users/3063408e632d405c955f7d233573bf73` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:42.553405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393c84e-1d8d-454b-8517-542c72db54a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3063408e632d405c955f7d233573bf73)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:42.555349 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393c84e-1d8d-454b-8517-542c72db54a6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:42.606321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3393c84e-1d8d-454b-8517-542c72db54a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3063408e632d405c955f7d233573bf73'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:42.607686 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3393c84e-1d8d-454b-8517-542c72db54a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3063408e632d405c955f7d233573bf73'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:42.612688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:42.617445 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3393c84e-1d8d-454b-8517-542c72db54a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3063408e632d405c955f7d233573bf73'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:42.633571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3393c84e-1d8d-454b-8517-542c72db54a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3063408e632d405c955f7d233573bf73'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:42.642210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3393c84e-1d8d-454b-8517-542c72db54a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3063408e632d405c955f7d233573bf73'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:42.648404 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1238/2456] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:46:42 2022] DELETE /identity/v3/users/3063408e632d405c955f7d233573bf73 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:42.929703 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:42.931253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:42.945268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a0396-f4da-45f8-98bd-f1a6075ecfb0 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:43.005631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a11faa-204b-414d-9159-6de2ba27560d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:43.006492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a11faa-204b-414d-9159-6de2ba27560d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:43.006768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a11faa-204b-414d-9159-6de2ba27560d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:43.007160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a11faa-204b-414d-9159-6de2ba27560d admin admin] PATH_INFO: `/v3/projects/5b7de874b82d4de0a6fe20737c9f99c2` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:43.008378 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a11faa-204b-414d-9159-6de2ba27560d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7de874b82d4de0a6fe20737c9f99c2)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:43.010299 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a11faa-204b-414d-9159-6de2ba27560d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:43.043302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:43.062587 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-71a11faa-204b-414d-9159-6de2ba27560d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7de874b82d4de0a6fe20737c9f99c2', 'request_id': 'req-71a11faa-204b-414d-9159-6de2ba27560d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:43.068914 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-71a11faa-204b-414d-9159-6de2ba27560d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7de874b82d4de0a6fe20737c9f99c2', 'request_id': 'req-71a11faa-204b-414d-9159-6de2ba27560d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:43.077701 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-71a11faa-204b-414d-9159-6de2ba27560d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7de874b82d4de0a6fe20737c9f99c2', 'request_id': 'req-71a11faa-204b-414d-9159-6de2ba27560d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:43.089299 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-71a11faa-204b-414d-9159-6de2ba27560d admin admin] The token cache is being invalidate because project 5b7de874b82d4de0a6fe20737c9f99c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:46:43.089694 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-71a11faa-204b-414d-9159-6de2ba27560d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:43.090559 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-71a11faa-204b-414d-9159-6de2ba27560d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:43.092837 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1219/2457] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:46:42 2022] DELETE /identity/v3/projects/5b7de874b82d4de0a6fe20737c9f99c2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:43.098112 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc3a6a-ea61-45a4-acc5-75ccf8b5004e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:43.098405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc3a6a-ea61-45a4-acc5-75ccf8b5004e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:43.098696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc3a6a-ea61-45a4-acc5-75ccf8b5004e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:43.116921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b2dc3a6a-ea61-45a4-acc5-75ccf8b5004e None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:43.118138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b2dc3a6a-ea61-45a4-acc5-75ccf8b5004e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:43.181482 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1239/2458] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:43.194265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca611bf-c5c2-4e6f-bc10-195c745e9312 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:43.194660 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca611bf-c5c2-4e6f-bc10-195c745e9312 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:43.194942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca611bf-c5c2-4e6f-bc10-195c745e9312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:43.214415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6ca611bf-c5c2-4e6f-bc10-195c745e9312 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:43.216560 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6ca611bf-c5c2-4e6f-bc10-195c745e9312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:43.225101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1220/2459] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:43.229376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc3a6a-ea61-45a4-acc5-75ccf8b5004e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:43.239204 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc3a6a-ea61-45a4-acc5-75ccf8b5004e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:43.243948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c1b3a-cd5c-4d93-9323-b89201a8a10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:43.244604 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c1b3a-cd5c-4d93-9323-b89201a8a10e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:43.244892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c1b3a-cd5c-4d93-9323-b89201a8a10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:43.245172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c1b3a-cd5c-4d93-9323-b89201a8a10e admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:43.245903 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c1b3a-cd5c-4d93-9323-b89201a8a10e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:46:43.246212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c1b3a-cd5c-4d93-9323-b89201a8a10e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:43.247871 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c1b3a-cd5c-4d93-9323-b89201a8a10e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:43.259700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1240/2460] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 09:46:43.264634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca611bf-c5c2-4e6f-bc10-195c745e9312 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:43.273986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca611bf-c5c2-4e6f-bc10-195c745e9312 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:43.280250 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeec047-3778-41e7-872f-75dc66fb54a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:43.280932 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeec047-3778-41e7-872f-75dc66fb54a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:43.281216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeec047-3778-41e7-872f-75dc66fb54a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:43.281490 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeec047-3778-41e7-872f-75dc66fb54a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:43.282179 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeec047-3778-41e7-872f-75dc66fb54a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:43.284170 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeec047-3778-41e7-872f-75dc66fb54a4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:43.309955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1221/2461] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:46:43 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:43.314521 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c1b3a-cd5c-4d93-9323-b89201a8a10e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:43.324489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c1b3a-cd5c-4d93-9323-b89201a8a10e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:43.381757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dafe87-247c-4b95-9df4-c6bddb72549d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:43.382439 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dafe87-247c-4b95-9df4-c6bddb72549d admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:43.382721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dafe87-247c-4b95-9df4-c6bddb72549d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:43.382995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dafe87-247c-4b95-9df4-c6bddb72549d admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:43.383694 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dafe87-247c-4b95-9df4-c6bddb72549d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:43.385641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dafe87-247c-4b95-9df4-c6bddb72549d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:43.406896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1241/2462] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:46:43 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:43.410953 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeec047-3778-41e7-872f-75dc66fb54a4 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:43.422259 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeec047-3778-41e7-872f-75dc66fb54a4 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:43.426913 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92499672-ab1d-4dca-956f-d84598c5c6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:43.427561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92499672-ab1d-4dca-956f-d84598c5c6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:43.427854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92499672-ab1d-4dca-956f-d84598c5c6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:43.428275 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92499672-ab1d-4dca-956f-d84598c5c6d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:43.428983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92499672-ab1d-4dca-956f-d84598c5c6d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:43.430597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92499672-ab1d-4dca-956f-d84598c5c6d2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:43.445677 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1222/2463] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:43.447197 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dafe87-247c-4b95-9df4-c6bddb72549d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:43.460453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dafe87-247c-4b95-9df4-c6bddb72549d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:43.466525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845e3df-415f-4edb-bc97-6c73b5094c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:43.467309 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845e3df-415f-4edb-bc97-6c73b5094c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:43.467648 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845e3df-415f-4edb-bc97-6c73b5094c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:43.468056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845e3df-415f-4edb-bc97-6c73b5094c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:43.468977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845e3df-415f-4edb-bc97-6c73b5094c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:43.470678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845e3df-415f-4edb-bc97-6c73b5094c40 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:43.484676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1242/2464] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:43.486212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92499672-ab1d-4dca-956f-d84598c5c6d2 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:43.496442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92499672-ab1d-4dca-956f-d84598c5c6d2 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:43.502389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c143a-9f7a-43b8-a615-962d78eebf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:43.503328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c143a-9f7a-43b8-a615-962d78eebf71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:43.503674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c143a-9f7a-43b8-a615-962d78eebf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:43.503991 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c143a-9f7a-43b8-a615-962d78eebf71 admin admin] PATH_INFO: `/v3/projects/5d9e68c922f84531a59986c221a11509/users/2b98fbe07c5c4d3db97fc4fcc8ba2012/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:43.520421 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c143a-9f7a-43b8-a615-962d78eebf71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9e68c922f84531a59986c221a11509, user_id=2b98fbe07c5c4d3db97fc4fcc8ba2012, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:43.523143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c143a-9f7a-43b8-a615-962d78eebf71 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:43.536209 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1223/2465] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:46:43 2022] PUT /identity/v3/projects/5d9e68c922f84531a59986c221a11509/users/2b98fbe07c5c4d3db97fc4fcc8ba2012/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:43.547545 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01139e9f-6d36-4cad-b3e8-df9882770115 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:43.547869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01139e9f-6d36-4cad-b3e8-df9882770115 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:43.548151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01139e9f-6d36-4cad-b3e8-df9882770115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:43.577037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-01139e9f-6d36-4cad-b3e8-df9882770115 None None] MFA Rules not processed for user `2b98fbe07c5c4d3db97fc4fcc8ba2012`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:43.578238 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-01139e9f-6d36-4cad-b3e8-df9882770115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:43.601820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:43.646565 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1243/2466] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:43.651256 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c143a-9f7a-43b8-a615-962d78eebf71 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:43.720715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c143a-9f7a-43b8-a615-962d78eebf71 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:43.725446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e9b52-65e2-4fea-99da-c79a9fbbfe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:43.726103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e9b52-65e2-4fea-99da-c79a9fbbfe99 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:43.726392 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e9b52-65e2-4fea-99da-c79a9fbbfe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:43.726663 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e9b52-65e2-4fea-99da-c79a9fbbfe99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:43.727395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e9b52-65e2-4fea-99da-c79a9fbbfe99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:43.729345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e9b52-65e2-4fea-99da-c79a9fbbfe99 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:43.757266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1224/2467] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:46:43 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:43.761457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01139e9f-6d36-4cad-b3e8-df9882770115 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:43.772728 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01139e9f-6d36-4cad-b3e8-df9882770115 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:43.831649 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4706b2-b5f6-4b29-8468-901d0794d0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:43.832339 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4706b2-b5f6-4b29-8468-901d0794d0d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:43.832622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4706b2-b5f6-4b29-8468-901d0794d0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:43.832895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4706b2-b5f6-4b29-8468-901d0794d0d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:43.833592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4706b2-b5f6-4b29-8468-901d0794d0d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:43.835508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4706b2-b5f6-4b29-8468-901d0794d0d0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:43.857822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1244/2468] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:46:43 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:43.862000 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e9b52-65e2-4fea-99da-c79a9fbbfe99 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:43.873041 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e9b52-65e2-4fea-99da-c79a9fbbfe99 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:43.877860 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698a8c9-51ab-4028-a6c9-d855367c8fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:43.878544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698a8c9-51ab-4028-a6c9-d855367c8fce admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:43.878807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698a8c9-51ab-4028-a6c9-d855367c8fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:43.879084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698a8c9-51ab-4028-a6c9-d855367c8fce admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:43.879816 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698a8c9-51ab-4028-a6c9-d855367c8fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:43.881532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698a8c9-51ab-4028-a6c9-d855367c8fce admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:43.892839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1225/2469] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:43.897158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4706b2-b5f6-4b29-8468-901d0794d0d0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:43.907241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4706b2-b5f6-4b29-8468-901d0794d0d0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:43.912294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19995784-7b15-48d3-bdb6-287a15e10bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:43.913182 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19995784-7b15-48d3-bdb6-287a15e10bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:43.913449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19995784-7b15-48d3-bdb6-287a15e10bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:43.913723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19995784-7b15-48d3-bdb6-287a15e10bc0 admin admin] PATH_INFO: `/v3/projects/d8ccbcc811694b4299178d276cfe6419/users/3d5e88ee21cc4550a800f02b1f2f7954/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:43.929896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19995784-7b15-48d3-bdb6-287a15e10bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8ccbcc811694b4299178d276cfe6419, user_id=3d5e88ee21cc4550a800f02b1f2f7954, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:43.932556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19995784-7b15-48d3-bdb6-287a15e10bc0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:43.945721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1245/2470] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:46:43 2022] PUT /identity/v3/projects/d8ccbcc811694b4299178d276cfe6419/users/3d5e88ee21cc4550a800f02b1f2f7954/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:43.949741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698a8c9-51ab-4028-a6c9-d855367c8fce admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:43.961079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698a8c9-51ab-4028-a6c9-d855367c8fce admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:43.982345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:44.024261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2524c58-09cb-4af0-a6cd-a64fae88aae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:44.024929 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2524c58-09cb-4af0-a6cd-a64fae88aae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:44.025212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2524c58-09cb-4af0-a6cd-a64fae88aae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:44.025489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2524c58-09cb-4af0-a6cd-a64fae88aae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:44.026213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2524c58-09cb-4af0-a6cd-a64fae88aae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:44.027835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2524c58-09cb-4af0-a6cd-a64fae88aae1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:44.038223 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1226/2471] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:44.042611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19995784-7b15-48d3-bdb6-287a15e10bc0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:44.054114 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19995784-7b15-48d3-bdb6-287a15e10bc0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:44.058945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cfff4e-7c61-4609-a2a8-6e08bd8f942c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:44.060555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfff4e-7c61-4609-a2a8-6e08bd8f942c admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:44.060912 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfff4e-7c61-4609-a2a8-6e08bd8f942c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:44.061290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfff4e-7c61-4609-a2a8-6e08bd8f942c admin admin] PATH_INFO: `/v3/projects/d8ccbcc811694b4299178d276cfe6419/users/3d5e88ee21cc4550a800f02b1f2f7954/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:44.063463 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cfff4e-7c61-4609-a2a8-6e08bd8f942c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8ccbcc811694b4299178d276cfe6419, user_id=3d5e88ee21cc4550a800f02b1f2f7954, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:44.066152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cfff4e-7c61-4609-a2a8-6e08bd8f942c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:44.077842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1246/2472] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:46:44 2022] PUT /identity/v3/projects/d8ccbcc811694b4299178d276cfe6419/users/3d5e88ee21cc4550a800f02b1f2f7954/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:44.089534 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272459dc-97a8-4d57-96bb-9d36889434c3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:44.089826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272459dc-97a8-4d57-96bb-9d36889434c3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:44.090105 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272459dc-97a8-4d57-96bb-9d36889434c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:44.118262 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-272459dc-97a8-4d57-96bb-9d36889434c3 None None] MFA Rules not processed for user `3d5e88ee21cc4550a800f02b1f2f7954`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:44.119455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-272459dc-97a8-4d57-96bb-9d36889434c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:44.191437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1227/2473] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:44 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:46:44.202905 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cfff4e-7c61-4609-a2a8-6e08bd8f942c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:44.204752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-83cfff4e-7c61-4609-a2a8-6e08bd8f942c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:44.267656 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cfff4e-7c61-4609-a2a8-6e08bd8f942c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:44.271267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c81a64-580f-4912-974e-ebc5c639f998 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:44.271902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c81a64-580f-4912-974e-ebc5c639f998 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:44.272219 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c81a64-580f-4912-974e-ebc5c639f998 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:44.272496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c81a64-580f-4912-974e-ebc5c639f998 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:44.282276 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c81a64-580f-4912-974e-ebc5c639f998 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:44.283700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c81a64-580f-4912-974e-ebc5c639f998 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:44.293017 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1247/2474] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:44 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:44.339203 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272459dc-97a8-4d57-96bb-9d36889434c3 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:44.340621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-272459dc-97a8-4d57-96bb-9d36889434c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:44.353064 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272459dc-97a8-4d57-96bb-9d36889434c3 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:44.357322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76519a0-b53e-46a9-8b9d-4cfbc74e1c31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:44.358106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76519a0-b53e-46a9-8b9d-4cfbc74e1c31 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:44.358392 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76519a0-b53e-46a9-8b9d-4cfbc74e1c31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:44.358675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76519a0-b53e-46a9-8b9d-4cfbc74e1c31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:44.414465 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76519a0-b53e-46a9-8b9d-4cfbc74e1c31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:44.415907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76519a0-b53e-46a9-8b9d-4cfbc74e1c31 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:44.425405 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1228/2475] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:45.768054 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c81a64-580f-4912-974e-ebc5c639f998 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:45.769563 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a4c81a64-580f-4912-974e-ebc5c639f998 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:45.792218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:45.840522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c81a64-580f-4912-974e-ebc5c639f998 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:45.846355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a42da6-8d5e-4285-aa23-7deee68d1961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:45.847243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a42da6-8d5e-4285-aa23-7deee68d1961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:45.847526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a42da6-8d5e-4285-aa23-7deee68d1961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:45.847798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a42da6-8d5e-4285-aa23-7deee68d1961 admin admin] PATH_INFO: `/v3/users/bd3b26480fc842d18aaa2f91a1b8fc60` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:45.849027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a42da6-8d5e-4285-aa23-7deee68d1961 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd3b26480fc842d18aaa2f91a1b8fc60)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:45.850991 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a42da6-8d5e-4285-aa23-7deee68d1961 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:45.915953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-53a42da6-8d5e-4285-aa23-7deee68d1961 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd3b26480fc842d18aaa2f91a1b8fc60'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:45.916949 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-53a42da6-8d5e-4285-aa23-7deee68d1961 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd3b26480fc842d18aaa2f91a1b8fc60'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:45.929400 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-53a42da6-8d5e-4285-aa23-7deee68d1961 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd3b26480fc842d18aaa2f91a1b8fc60'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:45.946005 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-53a42da6-8d5e-4285-aa23-7deee68d1961 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd3b26480fc842d18aaa2f91a1b8fc60'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:45.957351 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-53a42da6-8d5e-4285-aa23-7deee68d1961 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd3b26480fc842d18aaa2f91a1b8fc60'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:45.964906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1248/2476] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:46:45 2022] DELETE /identity/v3/users/bd3b26480fc842d18aaa2f91a1b8fc60 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:45.969856 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389b64a-0023-4045-95f4-758663b3bdb5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:45.970362 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389b64a-0023-4045-95f4-758663b3bdb5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:45.970469 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389b64a-0023-4045-95f4-758663b3bdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:45.989898 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d389b64a-0023-4045-95f4-758663b3bdb5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:45.991181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d389b64a-0023-4045-95f4-758663b3bdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:46.033394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:46.057401 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1229/2477] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:46.069086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a42da6-8d5e-4285-aa23-7deee68d1961 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:46.070681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-53a42da6-8d5e-4285-aa23-7deee68d1961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:46.136985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a42da6-8d5e-4285-aa23-7deee68d1961 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:46.142155 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421a703-59b5-4d3d-b4cd-f54bc16c783e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:46.142818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421a703-59b5-4d3d-b4cd-f54bc16c783e service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:46.143101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421a703-59b5-4d3d-b4cd-f54bc16c783e service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:46.143374 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421a703-59b5-4d3d-b4cd-f54bc16c783e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:46.154442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421a703-59b5-4d3d-b4cd-f54bc16c783e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:46.156411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421a703-59b5-4d3d-b4cd-f54bc16c783e service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:46.167102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1249/2478] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:46.452168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389b64a-0023-4045-95f4-758663b3bdb5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:46.453595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d389b64a-0023-4045-95f4-758663b3bdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:46.465556 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389b64a-0023-4045-95f4-758663b3bdb5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:46.470900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d07197-ac5d-47e3-aa14-325805bdac10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:46.471777 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d07197-ac5d-47e3-aa14-325805bdac10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:46.472075 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d07197-ac5d-47e3-aa14-325805bdac10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:46.472346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d07197-ac5d-47e3-aa14-325805bdac10 admin admin] PATH_INFO: `/v3/projects/d7473ead72c3435fa1a1f478b01a41d1` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:46.473520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d07197-ac5d-47e3-aa14-325805bdac10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7473ead72c3435fa1a1f478b01a41d1)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:46.475453 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d07197-ac5d-47e3-aa14-325805bdac10 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:46.531764 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-33d07197-ac5d-47e3-aa14-325805bdac10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7473ead72c3435fa1a1f478b01a41d1', 'request_id': 'req-33d07197-ac5d-47e3-aa14-325805bdac10'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:46.537855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-33d07197-ac5d-47e3-aa14-325805bdac10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7473ead72c3435fa1a1f478b01a41d1', 'request_id': 'req-33d07197-ac5d-47e3-aa14-325805bdac10'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:46.545515 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-33d07197-ac5d-47e3-aa14-325805bdac10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7473ead72c3435fa1a1f478b01a41d1', 'request_id': 'req-33d07197-ac5d-47e3-aa14-325805bdac10'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:46.556431 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-33d07197-ac5d-47e3-aa14-325805bdac10 admin admin] The token cache is being invalidate because project d7473ead72c3435fa1a1f478b01a41d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:46:46.556720 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-33d07197-ac5d-47e3-aa14-325805bdac10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:46.557351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-33d07197-ac5d-47e3-aa14-325805bdac10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:46.558948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1230/2479] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:46:46 2022] DELETE /identity/v3/projects/d7473ead72c3435fa1a1f478b01a41d1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:46.564513 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1128c78-107f-4529-a4f8-29c65b592fa3 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:46.564803 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1128c78-107f-4529-a4f8-29c65b592fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:46.565079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1128c78-107f-4529-a4f8-29c65b592fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:46.582808 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d1128c78-107f-4529-a4f8-29c65b592fa3 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:46.584053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d1128c78-107f-4529-a4f8-29c65b592fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:46.623966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:46.648484 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1250/2480] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:46.658255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c261d9-6084-4d58-9c21-cd8e62f104b1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:46.658592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c261d9-6084-4d58-9c21-cd8e62f104b1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:46.658877 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c261d9-6084-4d58-9c21-cd8e62f104b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:46.677884 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e3c261d9-6084-4d58-9c21-cd8e62f104b1 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:46.679111 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e3c261d9-6084-4d58-9c21-cd8e62f104b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:46.688925 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1231/2481] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:46.693520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1128c78-107f-4529-a4f8-29c65b592fa3 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:46.703500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1128c78-107f-4529-a4f8-29c65b592fa3 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:46.708186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8301cfc-dd44-40bc-9931-2a970ef3c261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:46.708837 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8301cfc-dd44-40bc-9931-2a970ef3c261 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:46.709122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8301cfc-dd44-40bc-9931-2a970ef3c261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:46.709425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8301cfc-dd44-40bc-9931-2a970ef3c261 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:46.710165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8301cfc-dd44-40bc-9931-2a970ef3c261 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:46:46.710470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8301cfc-dd44-40bc-9931-2a970ef3c261 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:46.712118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8301cfc-dd44-40bc-9931-2a970ef3c261 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:46.727522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1251/2482] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:46:46.732714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c261d9-6084-4d58-9c21-cd8e62f104b1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:46.742214 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c261d9-6084-4d58-9c21-cd8e62f104b1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:46.747047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff480603-7e16-436b-b6e8-a8839253c56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:46.747817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff480603-7e16-436b-b6e8-a8839253c56e admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:46.748153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff480603-7e16-436b-b6e8-a8839253c56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:46.748434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff480603-7e16-436b-b6e8-a8839253c56e admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:46.749143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff480603-7e16-436b-b6e8-a8839253c56e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:46.751099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff480603-7e16-436b-b6e8-a8839253c56e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:46.777344 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1232/2483] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:46:46 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:46.782157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8301cfc-dd44-40bc-9931-2a970ef3c261 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:46.792154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8301cfc-dd44-40bc-9931-2a970ef3c261 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:46.850116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83d550-1b32-4778-8c5b-c3a777bc4700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:46.850775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83d550-1b32-4778-8c5b-c3a777bc4700 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:46.851058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83d550-1b32-4778-8c5b-c3a777bc4700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:46.851331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83d550-1b32-4778-8c5b-c3a777bc4700 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:46.852175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83d550-1b32-4778-8c5b-c3a777bc4700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:46.854089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83d550-1b32-4778-8c5b-c3a777bc4700 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:46.873914 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1252/2484] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:46:46 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:46.878107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff480603-7e16-436b-b6e8-a8839253c56e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:46.889089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff480603-7e16-436b-b6e8-a8839253c56e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:46.893758 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d95e0-02d3-4175-bca7-d88985123a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:46.894414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d95e0-02d3-4175-bca7-d88985123a9f admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:46.894702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d95e0-02d3-4175-bca7-d88985123a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:46.894979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d95e0-02d3-4175-bca7-d88985123a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:46.895702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d95e0-02d3-4175-bca7-d88985123a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:46.897359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d95e0-02d3-4175-bca7-d88985123a9f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:46.908584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1233/2485] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:46.912651 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83d550-1b32-4778-8c5b-c3a777bc4700 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:46.923044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83d550-1b32-4778-8c5b-c3a777bc4700 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:46.927595 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c554543-db2e-42be-ad63-253f941f8be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:46.928273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c554543-db2e-42be-ad63-253f941f8be2 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:46.928555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c554543-db2e-42be-ad63-253f941f8be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:46.928826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c554543-db2e-42be-ad63-253f941f8be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:46.929544 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c554543-db2e-42be-ad63-253f941f8be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:46.931163 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c554543-db2e-42be-ad63-253f941f8be2 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:46.942106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1253/2486] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:46.946279 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d95e0-02d3-4175-bca7-d88985123a9f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:46.956936 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d95e0-02d3-4175-bca7-d88985123a9f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:46.961439 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190e85e-05e4-4507-8017-6fe9824cee2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:46.962302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e85e-05e4-4507-8017-6fe9824cee2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:46.962586 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e85e-05e4-4507-8017-6fe9824cee2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:46.962859 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e85e-05e4-4507-8017-6fe9824cee2b admin admin] PATH_INFO: `/v3/projects/50c0c53ad76c48e7bf856c4fd07f187e/users/d5708973a51c49b39b741ec63e4b21ad/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:46.978195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190e85e-05e4-4507-8017-6fe9824cee2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c0c53ad76c48e7bf856c4fd07f187e, user_id=d5708973a51c49b39b741ec63e4b21ad, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:46.980901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190e85e-05e4-4507-8017-6fe9824cee2b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:46.994495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1234/2487] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:46:46 2022] PUT /identity/v3/projects/50c0c53ad76c48e7bf856c4fd07f187e/users/d5708973a51c49b39b741ec63e4b21ad/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:47.007055 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc0cbd-a4be-4dc6-acaf-887d84bf00d9 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:47.007055 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc0cbd-a4be-4dc6-acaf-887d84bf00d9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:47.007055 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc0cbd-a4be-4dc6-acaf-887d84bf00d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:47.034905 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-95bc0cbd-a4be-4dc6-acaf-887d84bf00d9 None None] MFA Rules not processed for user `d5708973a51c49b39b741ec63e4b21ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:47.036170 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-95bc0cbd-a4be-4dc6-acaf-887d84bf00d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:47.109355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1254/2488] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:47.120688 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190e85e-05e4-4507-8017-6fe9824cee2b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:47.123045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f190e85e-05e4-4507-8017-6fe9824cee2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:47.180057 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190e85e-05e4-4507-8017-6fe9824cee2b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:47.183703 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6cec8-2b7b-46d4-b84c-e9d78898eac3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:47.184370 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6cec8-2b7b-46d4-b84c-e9d78898eac3 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:47.185257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6cec8-2b7b-46d4-b84c-e9d78898eac3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:47.185543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6cec8-2b7b-46d4-b84c-e9d78898eac3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:47.193790 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6cec8-2b7b-46d4-b84c-e9d78898eac3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:47.195239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6cec8-2b7b-46d4-b84c-e9d78898eac3 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:47.204982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1235/2489] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:47 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:47.229265 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc0cbd-a4be-4dc6-acaf-887d84bf00d9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:47.277512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:47.296035 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc0cbd-a4be-4dc6-acaf-887d84bf00d9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:47.300849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa692eb6-57a0-413c-a50b-42843ed73476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:47.301749 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa692eb6-57a0-413c-a50b-42843ed73476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:47.302028 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa692eb6-57a0-413c-a50b-42843ed73476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:47.302300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa692eb6-57a0-413c-a50b-42843ed73476 admin admin] PATH_INFO: `/v3/users/d5708973a51c49b39b741ec63e4b21ad` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:47.303549 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa692eb6-57a0-413c-a50b-42843ed73476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5708973a51c49b39b741ec63e4b21ad)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:47.305583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa692eb6-57a0-413c-a50b-42843ed73476 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:47.355118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-aa692eb6-57a0-413c-a50b-42843ed73476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5708973a51c49b39b741ec63e4b21ad'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:47.356448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-aa692eb6-57a0-413c-a50b-42843ed73476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5708973a51c49b39b741ec63e4b21ad'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:47.364575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-aa692eb6-57a0-413c-a50b-42843ed73476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5708973a51c49b39b741ec63e4b21ad'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:47.385322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-aa692eb6-57a0-413c-a50b-42843ed73476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5708973a51c49b39b741ec63e4b21ad'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:47.395084 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-aa692eb6-57a0-413c-a50b-42843ed73476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5708973a51c49b39b741ec63e4b21ad'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:47.405663 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1255/2490] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:46:47 2022] DELETE /identity/v3/users/d5708973a51c49b39b741ec63e4b21ad => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:47.410297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba2726-8d0d-428c-aa07-8af394ec31e4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:47.410589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba2726-8d0d-428c-aa07-8af394ec31e4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:47.410865 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba2726-8d0d-428c-aa07-8af394ec31e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:47.433374 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:47.440983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-30ba2726-8d0d-428c-aa07-8af394ec31e4 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:47.442751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-30ba2726-8d0d-428c-aa07-8af394ec31e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:47.513335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1236/2491] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:47.523063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa692eb6-57a0-413c-a50b-42843ed73476 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:47.524599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-aa692eb6-57a0-413c-a50b-42843ed73476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:47.586613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa692eb6-57a0-413c-a50b-42843ed73476 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:47.590488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c2484-9a75-4158-b75c-757464d08406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:47.591781 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c2484-9a75-4158-b75c-757464d08406 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:47.592110 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c2484-9a75-4158-b75c-757464d08406 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:47.592382 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c2484-9a75-4158-b75c-757464d08406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:47.602654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c2484-9a75-4158-b75c-757464d08406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:47.603989 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c2484-9a75-4158-b75c-757464d08406 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:47.614144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1256/2492] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:47.830546 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba2726-8d0d-428c-aa07-8af394ec31e4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:47.832003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-30ba2726-8d0d-428c-aa07-8af394ec31e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:47.844510 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba2726-8d0d-428c-aa07-8af394ec31e4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:47.849209 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6f59e-0059-432f-bfc2-1d40c36a939e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:47.850605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6f59e-0059-432f-bfc2-1d40c36a939e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:47.850605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6f59e-0059-432f-bfc2-1d40c36a939e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:47.850605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6f59e-0059-432f-bfc2-1d40c36a939e admin admin] PATH_INFO: `/v3/users/2b98fbe07c5c4d3db97fc4fcc8ba2012` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:47.852169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad6f59e-0059-432f-bfc2-1d40c36a939e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b98fbe07c5c4d3db97fc4fcc8ba2012)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:47.854136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad6f59e-0059-432f-bfc2-1d40c36a939e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:47.905060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4ad6f59e-0059-432f-bfc2-1d40c36a939e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b98fbe07c5c4d3db97fc4fcc8ba2012'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:47.912932 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4ad6f59e-0059-432f-bfc2-1d40c36a939e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b98fbe07c5c4d3db97fc4fcc8ba2012'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:47.920808 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c2484-9a75-4158-b75c-757464d08406 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:47.922535 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-269c2484-9a75-4158-b75c-757464d08406 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:47.925033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4ad6f59e-0059-432f-bfc2-1d40c36a939e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b98fbe07c5c4d3db97fc4fcc8ba2012'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:47.940300 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4ad6f59e-0059-432f-bfc2-1d40c36a939e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b98fbe07c5c4d3db97fc4fcc8ba2012'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:47.941118 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4ad6f59e-0059-432f-bfc2-1d40c36a939e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b98fbe07c5c4d3db97fc4fcc8ba2012'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:47.947237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1237/2493] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:46:47 2022] DELETE /identity/v3/users/2b98fbe07c5c4d3db97fc4fcc8ba2012 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:47.950018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c2484-9a75-4158-b75c-757464d08406 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:47.953062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946f51a-73da-4c26-9f96-9bc5cf081adf None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:47.953401 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946f51a-73da-4c26-9f96-9bc5cf081adf None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:47.953696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946f51a-73da-4c26-9f96-9bc5cf081adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:47.972889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e946f51a-73da-4c26-9f96-9bc5cf081adf None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:47.974018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e946f51a-73da-4c26-9f96-9bc5cf081adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:48.017121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcee10e-83d7-4f3e-a142-dbe93c8fd504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:48.017992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcee10e-83d7-4f3e-a142-dbe93c8fd504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:48.018306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcee10e-83d7-4f3e-a142-dbe93c8fd504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:48.018584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcee10e-83d7-4f3e-a142-dbe93c8fd504 admin admin] PATH_INFO: `/v3/projects/50c0c53ad76c48e7bf856c4fd07f187e` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:48.019795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcee10e-83d7-4f3e-a142-dbe93c8fd504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50c0c53ad76c48e7bf856c4fd07f187e)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:48.021787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcee10e-83d7-4f3e-a142-dbe93c8fd504 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:48.023408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:48.037672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1238/2494] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:48.048533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946f51a-73da-4c26-9f96-9bc5cf081adf None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:48.049963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e946f51a-73da-4c26-9f96-9bc5cf081adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:48.072730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:48.076530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bdcee10e-83d7-4f3e-a142-dbe93c8fd504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50c0c53ad76c48e7bf856c4fd07f187e', 'request_id': 'req-bdcee10e-83d7-4f3e-a142-dbe93c8fd504'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:48.082341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bdcee10e-83d7-4f3e-a142-dbe93c8fd504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50c0c53ad76c48e7bf856c4fd07f187e', 'request_id': 'req-bdcee10e-83d7-4f3e-a142-dbe93c8fd504'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:48.089368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bdcee10e-83d7-4f3e-a142-dbe93c8fd504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50c0c53ad76c48e7bf856c4fd07f187e', 'request_id': 'req-bdcee10e-83d7-4f3e-a142-dbe93c8fd504'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:48.105662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bdcee10e-83d7-4f3e-a142-dbe93c8fd504 admin admin] The token cache is being invalidate because project 50c0c53ad76c48e7bf856c4fd07f187e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:46:48.105767 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bdcee10e-83d7-4f3e-a142-dbe93c8fd504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:48.106520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bdcee10e-83d7-4f3e-a142-dbe93c8fd504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:48.111984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1257/2495] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:46:47 2022] DELETE /identity/v3/projects/50c0c53ad76c48e7bf856c4fd07f187e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:46:48.113564 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f872a-5642-413d-b379-357e3db6ee1f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:48.113852 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f872a-5642-413d-b379-357e3db6ee1f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:48.114156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f872a-5642-413d-b379-357e3db6ee1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:48.118480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946f51a-73da-4c26-9f96-9bc5cf081adf None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:48.140826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-794f872a-5642-413d-b379-357e3db6ee1f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:48.142047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-794f872a-5642-413d-b379-357e3db6ee1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:48.182447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef3a4f-0176-4fc5-987b-eb5c22eb2c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:48.183106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef3a4f-0176-4fc5-987b-eb5c22eb2c39 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:48.183521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef3a4f-0176-4fc5-987b-eb5c22eb2c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:48.183664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef3a4f-0176-4fc5-987b-eb5c22eb2c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:48.186647 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-58ef3a4f-0176-4fc5-987b-eb5c22eb2c39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:48.209352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1258/2496] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:48.217844 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af7b81-bf4e-4d76-ae3f-5543ff28a7a5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:48.218135 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af7b81-bf4e-4d76-ae3f-5543ff28a7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:48.218414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af7b81-bf4e-4d76-ae3f-5543ff28a7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:48.237897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-44af7b81-bf4e-4d76-ae3f-5543ff28a7a5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:48.239075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-44af7b81-bf4e-4d76-ae3f-5543ff28a7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:48.247871 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1259/2497] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:46:48.252624 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af7b81-bf4e-4d76-ae3f-5543ff28a7a5 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:48.264640 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af7b81-bf4e-4d76-ae3f-5543ff28a7a5 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:48.271504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d9bca-42c2-4475-a48d-ba0b18f7e05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:48.272285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d9bca-42c2-4475-a48d-ba0b18f7e05e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:48.272639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d9bca-42c2-4475-a48d-ba0b18f7e05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:48.272980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d9bca-42c2-4475-a48d-ba0b18f7e05e admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:48.273786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d9bca-42c2-4475-a48d-ba0b18f7e05e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:46:48.274186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d9bca-42c2-4475-a48d-ba0b18f7e05e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:48.275100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef3a4f-0176-4fc5-987b-eb5c22eb2c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:48.276239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d9bca-42c2-4475-a48d-ba0b18f7e05e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:48.276604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef3a4f-0176-4fc5-987b-eb5c22eb2c39 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:48.288412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1239/2498] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:46:48.291440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1260/2499] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 09:46:48.297186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef3a4f-0176-4fc5-987b-eb5c22eb2c39 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:48.315674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef3a4f-0176-4fc5-987b-eb5c22eb2c39 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:48.320478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9d3f6-7840-44f0-bf7d-a498a72ed3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:48.321140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9d3f6-7840-44f0-bf7d-a498a72ed3ce admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:48.321466 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9d3f6-7840-44f0-bf7d-a498a72ed3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:48.321742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9d3f6-7840-44f0-bf7d-a498a72ed3ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:48.322477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9d3f6-7840-44f0-bf7d-a498a72ed3ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:48.324483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9d3f6-7840-44f0-bf7d-a498a72ed3ce admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:48.353193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1240/2500] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:46:48 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:48.359119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d9bca-42c2-4475-a48d-ba0b18f7e05e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:48.370485 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d9bca-42c2-4475-a48d-ba0b18f7e05e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:48.433601 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd92b3-63ba-42ca-b4de-4684b96d16ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:48.434302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd92b3-63ba-42ca-b4de-4684b96d16ea admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:48.434621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd92b3-63ba-42ca-b4de-4684b96d16ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:48.436287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd92b3-63ba-42ca-b4de-4684b96d16ea admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:48.436619 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd92b3-63ba-42ca-b4de-4684b96d16ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:48.439946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd92b3-63ba-42ca-b4de-4684b96d16ea admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:48.460225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:48.472480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1261/2501] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:46:48 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:46:48.476835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9d3f6-7840-44f0-bf7d-a498a72ed3ce admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:48.489107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9d3f6-7840-44f0-bf7d-a498a72ed3ce admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:48.495040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a9f5f-9545-412f-af48-39cce3db7468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:48.495949 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a9f5f-9545-412f-af48-39cce3db7468 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:48.496234 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a9f5f-9545-412f-af48-39cce3db7468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:48.496507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a9f5f-9545-412f-af48-39cce3db7468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:48.497951 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a9f5f-9545-412f-af48-39cce3db7468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:48.499962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a9f5f-9545-412f-af48-39cce3db7468 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:48.515200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1241/2502] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:48.516877 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd92b3-63ba-42ca-b4de-4684b96d16ea admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:48.527623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd92b3-63ba-42ca-b4de-4684b96d16ea admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:48.532996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b3229-1399-4b02-bcc2-5e04e6cff673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:48.533830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b3229-1399-4b02-bcc2-5e04e6cff673 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:48.534234 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b3229-1399-4b02-bcc2-5e04e6cff673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:48.534621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b3229-1399-4b02-bcc2-5e04e6cff673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:48.535495 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b3229-1399-4b02-bcc2-5e04e6cff673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:48.537461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b3229-1399-4b02-bcc2-5e04e6cff673 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:48.553383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1262/2503] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:48.555240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a9f5f-9545-412f-af48-39cce3db7468 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:48.567321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a9f5f-9545-412f-af48-39cce3db7468 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:48.572491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1c515-b665-482e-b618-5442757af690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:48.573375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1c515-b665-482e-b618-5442757af690 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:48.573660 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1c515-b665-482e-b618-5442757af690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:48.573938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1c515-b665-482e-b618-5442757af690 admin admin] PATH_INFO: `/v3/projects/63584d9f32ac4da0835773fec871e121/users/d23a2825a06e4297a96ab098af99cd5c/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:48.591337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1c515-b665-482e-b618-5442757af690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63584d9f32ac4da0835773fec871e121, user_id=d23a2825a06e4297a96ab098af99cd5c, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:48.594388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1c515-b665-482e-b618-5442757af690 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:48.603278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b3229-1399-4b02-bcc2-5e04e6cff673 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:48.605695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-625b3229-1399-4b02-bcc2-5e04e6cff673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:48.617423 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1242/2504] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:46:48 2022] PUT /identity/v3/projects/63584d9f32ac4da0835773fec871e121/users/d23a2825a06e4297a96ab098af99cd5c/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:48.628671 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b3229-1399-4b02-bcc2-5e04e6cff673 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:48.633098 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4b480-97d9-44d4-82b3-997f0941116c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:48.633440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4b480-97d9-44d4-82b3-997f0941116c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:48.633812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4b480-97d9-44d4-82b3-997f0941116c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:48.668773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-9fa4b480-97d9-44d4-82b3-997f0941116c None None] MFA Rules not processed for user `d23a2825a06e4297a96ab098af99cd5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:48.671237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9fa4b480-97d9-44d4-82b3-997f0941116c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:48.688386 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:48.694990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:48.695862 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:48.696181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:48.696465 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7 admin admin] PATH_INFO: `/v3/projects/5d9e68c922f84531a59986c221a11509` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:48.697689 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d9e68c922f84531a59986c221a11509)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:48.699628 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:48.752201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1243/2505] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:46:48.763627 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4b480-97d9-44d4-82b3-997f0941116c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:48.765076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9fa4b480-97d9-44d4-82b3-997f0941116c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:48.769118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d9e68c922f84531a59986c221a11509', 'request_id': 'req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:48.775154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d9e68c922f84531a59986c221a11509', 'request_id': 'req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:48.782778 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d9e68c922f84531a59986c221a11509', 'request_id': 'req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:48.796472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7 admin admin] The token cache is being invalidate because project 5d9e68c922f84531a59986c221a11509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:46:48.796776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:48.797425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:48.799024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1263/2506] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:46:48 2022] DELETE /identity/v3/projects/5d9e68c922f84531a59986c221a11509 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:48.802964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:48.804355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:48.828826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4b480-97d9-44d4-82b3-997f0941116c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:48.871118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbe532-bed6-4014-8c07-2fc0146ae5e7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:48.878305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cd5e2-96d4-47d8-b45a-9900925f135f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:48.879318 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cd5e2-96d4-47d8-b45a-9900925f135f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:48.879655 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cd5e2-96d4-47d8-b45a-9900925f135f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:48.879898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cd5e2-96d4-47d8-b45a-9900925f135f admin admin] PATH_INFO: `/v3/users/3d5e88ee21cc4550a800f02b1f2f7954` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:48.881110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590438e-d14b-4db9-8156-50afb25bded8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:48.881316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cd5e2-96d4-47d8-b45a-9900925f135f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d5e88ee21cc4550a800f02b1f2f7954)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:48.881788 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590438e-d14b-4db9-8156-50afb25bded8 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:48.882072 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590438e-d14b-4db9-8156-50afb25bded8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:48.882445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590438e-d14b-4db9-8156-50afb25bded8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:48.883298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cd5e2-96d4-47d8-b45a-9900925f135f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:48.884273 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d590438e-d14b-4db9-8156-50afb25bded8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:48.909344 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:48.938401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6d5cd5e2-96d4-47d8-b45a-9900925f135f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d5e88ee21cc4550a800f02b1f2f7954'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:48.939108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6d5cd5e2-96d4-47d8-b45a-9900925f135f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d5e88ee21cc4550a800f02b1f2f7954'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:48.948018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6d5cd5e2-96d4-47d8-b45a-9900925f135f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d5e88ee21cc4550a800f02b1f2f7954'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:48.953743 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590438e-d14b-4db9-8156-50afb25bded8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:48.955331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d590438e-d14b-4db9-8156-50afb25bded8 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:48.963464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6d5cd5e2-96d4-47d8-b45a-9900925f135f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d5e88ee21cc4550a800f02b1f2f7954'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:48.966515 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1244/2507] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:46:48.972890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6d5cd5e2-96d4-47d8-b45a-9900925f135f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d5e88ee21cc4550a800f02b1f2f7954'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:48.980329 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1264/2508] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:46:48 2022] DELETE /identity/v3/users/3d5e88ee21cc4550a800f02b1f2f7954 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:46:49.320084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590438e-d14b-4db9-8156-50afb25bded8 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:49.346021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d590438e-d14b-4db9-8156-50afb25bded8 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:49.386420 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:49.429127 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39de3b-a62b-40a2-9b3d-2d7130be810c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:49.430005 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39de3b-a62b-40a2-9b3d-2d7130be810c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:49.430326 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39de3b-a62b-40a2-9b3d-2d7130be810c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:49.430758 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39de3b-a62b-40a2-9b3d-2d7130be810c admin admin] PATH_INFO: `/v3/projects/d8ccbcc811694b4299178d276cfe6419` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:49.432129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39de3b-a62b-40a2-9b3d-2d7130be810c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8ccbcc811694b4299178d276cfe6419)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:49.434876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39de3b-a62b-40a2-9b3d-2d7130be810c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:49.496776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3d39de3b-a62b-40a2-9b3d-2d7130be810c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8ccbcc811694b4299178d276cfe6419', 'request_id': 'req-3d39de3b-a62b-40a2-9b3d-2d7130be810c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:49.504507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3d39de3b-a62b-40a2-9b3d-2d7130be810c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8ccbcc811694b4299178d276cfe6419', 'request_id': 'req-3d39de3b-a62b-40a2-9b3d-2d7130be810c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:49.514366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3d39de3b-a62b-40a2-9b3d-2d7130be810c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8ccbcc811694b4299178d276cfe6419', 'request_id': 'req-3d39de3b-a62b-40a2-9b3d-2d7130be810c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:49.525652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3d39de3b-a62b-40a2-9b3d-2d7130be810c admin admin] The token cache is being invalidate because project d8ccbcc811694b4299178d276cfe6419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:46:49.525947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3d39de3b-a62b-40a2-9b3d-2d7130be810c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:49.526640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3d39de3b-a62b-40a2-9b3d-2d7130be810c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:49.528802 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1245/2509] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:46:49 2022] DELETE /identity/v3/projects/d8ccbcc811694b4299178d276cfe6419 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:49.534476 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb379e-9e6b-456a-b468-eeb221ea94e2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:49.536111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb379e-9e6b-456a-b468-eeb221ea94e2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:49.536111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb379e-9e6b-456a-b468-eeb221ea94e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:49.555567 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3dcb379e-9e6b-456a-b468-eeb221ea94e2 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:49.556812 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3dcb379e-9e6b-456a-b468-eeb221ea94e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:49.619553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1265/2510] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:49.628043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e8e7d-6fc5-46d0-8a97-2f7015794dee None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:49.628336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e8e7d-6fc5-46d0-8a97-2f7015794dee None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:49.628614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e8e7d-6fc5-46d0-8a97-2f7015794dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:49.649002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0c7e8e7d-6fc5-46d0-8a97-2f7015794dee None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:49.650200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0c7e8e7d-6fc5-46d0-8a97-2f7015794dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:49.659081 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1246/2511] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:49.663626 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb379e-9e6b-456a-b468-eeb221ea94e2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:49.674404 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb379e-9e6b-456a-b468-eeb221ea94e2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:49.679092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8239d-b023-40ef-9350-16f547b20a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:49.679741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8239d-b023-40ef-9350-16f547b20a97 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:49.680073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8239d-b023-40ef-9350-16f547b20a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:49.680352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8239d-b023-40ef-9350-16f547b20a97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:49.681088 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8239d-b023-40ef-9350-16f547b20a97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:46:49.681399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8239d-b023-40ef-9350-16f547b20a97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:49.683038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8239d-b023-40ef-9350-16f547b20a97 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:49.696236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1266/2512] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:46:49.701333 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e8e7d-6fc5-46d0-8a97-2f7015794dee None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:49.713784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e8e7d-6fc5-46d0-8a97-2f7015794dee None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:49.720738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604b013-044e-485e-892d-a5cce3af1dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:49.721415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604b013-044e-485e-892d-a5cce3af1dbb admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:49.721702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604b013-044e-485e-892d-a5cce3af1dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:49.721978 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604b013-044e-485e-892d-a5cce3af1dbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:49.722686 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604b013-044e-485e-892d-a5cce3af1dbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:49.724704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604b013-044e-485e-892d-a5cce3af1dbb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:49.753073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1247/2513] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:46:49 2022] POST /identity/v3/projects => generated 401 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:49.757625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8239d-b023-40ef-9350-16f547b20a97 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:49.769218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8239d-b023-40ef-9350-16f547b20a97 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:49.831197 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4b07e-9353-4736-ac2a-6860ab133cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:49.832086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4b07e-9353-4736-ac2a-6860ab133cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:49.832378 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4b07e-9353-4736-ac2a-6860ab133cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:49.832772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4b07e-9353-4736-ac2a-6860ab133cf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:49.833513 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4b07e-9353-4736-ac2a-6860ab133cf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:49.835670 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4b07e-9353-4736-ac2a-6860ab133cf5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:49.859156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1267/2514] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:46:49 2022] POST /identity/v3/users => generated 301 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:46:49.863644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604b013-044e-485e-892d-a5cce3af1dbb admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:49.874606 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604b013-044e-485e-892d-a5cce3af1dbb admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:49.879410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ec6a3-fc48-4517-9f84-df87fced8c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:49.880106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ec6a3-fc48-4517-9f84-df87fced8c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:49.880396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ec6a3-fc48-4517-9f84-df87fced8c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:49.880674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ec6a3-fc48-4517-9f84-df87fced8c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:49.881404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ec6a3-fc48-4517-9f84-df87fced8c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:49.883025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ec6a3-fc48-4517-9f84-df87fced8c57 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:49.894454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1248/2515] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:46:49.898578 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4b07e-9353-4736-ac2a-6860ab133cf5 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:49.909681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4b07e-9353-4736-ac2a-6860ab133cf5 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:49.914534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59c68d-54fa-428d-bb51-2a7874dc2c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:49.915192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59c68d-54fa-428d-bb51-2a7874dc2c02 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:49.915484 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59c68d-54fa-428d-bb51-2a7874dc2c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:49.915763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59c68d-54fa-428d-bb51-2a7874dc2c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:49.916540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59c68d-54fa-428d-bb51-2a7874dc2c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:49.918153 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59c68d-54fa-428d-bb51-2a7874dc2c02 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:49.937183 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1268/2516] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:49.938687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ec6a3-fc48-4517-9f84-df87fced8c57 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:49.949361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ec6a3-fc48-4517-9f84-df87fced8c57 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:49.954773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf0304-83df-4144-8481-5cfc1e26379f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:49.955703 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf0304-83df-4144-8481-5cfc1e26379f admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:49.956145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf0304-83df-4144-8481-5cfc1e26379f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:49.956641 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf0304-83df-4144-8481-5cfc1e26379f admin admin] PATH_INFO: `/v3/projects/5cf8330933a3474abcfdce0d65186f2e/users/d200fb43b3c448d49e85e03f5eaab88e/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:49.972827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf0304-83df-4144-8481-5cfc1e26379f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cf8330933a3474abcfdce0d65186f2e, user_id=d200fb43b3c448d49e85e03f5eaab88e, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:49.975536 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf0304-83df-4144-8481-5cfc1e26379f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:49.989128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1249/2517] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:46:49 2022] PUT /identity/v3/projects/5cf8330933a3474abcfdce0d65186f2e/users/d200fb43b3c448d49e85e03f5eaab88e/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:46:49.999745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9291f0-0353-4950-b0cf-4aa5c07bc97d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:50.000289 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9291f0-0353-4950-b0cf-4aa5c07bc97d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:50.000428 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9291f0-0353-4950-b0cf-4aa5c07bc97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:50.007070 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:50.030812 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4d9291f0-0353-4950-b0cf-4aa5c07bc97d None None] MFA Rules not processed for user `d200fb43b3c448d49e85e03f5eaab88e`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:50.031996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4d9291f0-0353-4950-b0cf-4aa5c07bc97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:50.097700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1269/2518] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:50.102251 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf0304-83df-4144-8481-5cfc1e26379f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:50.159352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:50.165266 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf0304-83df-4144-8481-5cfc1e26379f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:50.172136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907a1de-109e-4ea8-89ca-e52112d6c294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:50.172944 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907a1de-109e-4ea8-89ca-e52112d6c294 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:50.173284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907a1de-109e-4ea8-89ca-e52112d6c294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:50.173735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907a1de-109e-4ea8-89ca-e52112d6c294 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:50.174427 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907a1de-109e-4ea8-89ca-e52112d6c294 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:50.176418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907a1de-109e-4ea8-89ca-e52112d6c294 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:50.202691 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1250/2519] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:46:50 2022] POST /identity/v3/projects => generated 401 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:50.206804 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9291f0-0353-4950-b0cf-4aa5c07bc97d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:50.217407 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9291f0-0353-4950-b0cf-4aa5c07bc97d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:50.274158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe34dc-77e7-4243-b338-82731562a361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:50.274836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe34dc-77e7-4243-b338-82731562a361 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:50.275123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe34dc-77e7-4243-b338-82731562a361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:50.275401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe34dc-77e7-4243-b338-82731562a361 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:50.276129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe34dc-77e7-4243-b338-82731562a361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:50.278074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe34dc-77e7-4243-b338-82731562a361 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:50.299172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1270/2520] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:46:50 2022] POST /identity/v3/users => generated 307 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:50.303410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907a1de-109e-4ea8-89ca-e52112d6c294 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:50.315336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907a1de-109e-4ea8-89ca-e52112d6c294 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:50.320104 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c2216-bbb1-4d5f-a699-f9409bf5ca7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:50.320772 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c2216-bbb1-4d5f-a699-f9409bf5ca7e admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:50.321067 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c2216-bbb1-4d5f-a699-f9409bf5ca7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:50.321339 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c2216-bbb1-4d5f-a699-f9409bf5ca7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:50.322100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c2216-bbb1-4d5f-a699-f9409bf5ca7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:50.323734 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c2216-bbb1-4d5f-a699-f9409bf5ca7e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:50.336017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1251/2521] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:50.340212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe34dc-77e7-4243-b338-82731562a361 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:50.350568 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe34dc-77e7-4243-b338-82731562a361 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:50.355599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bec9eb-64b0-4778-851a-be9c1ad32881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:50.356671 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bec9eb-64b0-4778-851a-be9c1ad32881 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:50.357138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bec9eb-64b0-4778-851a-be9c1ad32881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:50.357410 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bec9eb-64b0-4778-851a-be9c1ad32881 admin admin] PATH_INFO: `/v3/projects/cdd3f75984b349a0b33deb11fae51f27/users/2d7dfb44d22a42d4b6df435b03fe08a8/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:50.374761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bec9eb-64b0-4778-851a-be9c1ad32881 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdd3f75984b349a0b33deb11fae51f27, user_id=2d7dfb44d22a42d4b6df435b03fe08a8, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:50.377545 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bec9eb-64b0-4778-851a-be9c1ad32881 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:50.392137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1271/2522] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:46:50 2022] PUT /identity/v3/projects/cdd3f75984b349a0b33deb11fae51f27/users/2d7dfb44d22a42d4b6df435b03fe08a8/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:46:50.396296 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c2216-bbb1-4d5f-a699-f9409bf5ca7e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:50.406349 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c2216-bbb1-4d5f-a699-f9409bf5ca7e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:50.464363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26061dd5-8b6c-4a8b-a104-809f5b75549e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:50.465019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26061dd5-8b6c-4a8b-a104-809f5b75549e admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:50.465304 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26061dd5-8b6c-4a8b-a104-809f5b75549e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:50.465766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26061dd5-8b6c-4a8b-a104-809f5b75549e admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:50.466799 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26061dd5-8b6c-4a8b-a104-809f5b75549e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:50.468450 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26061dd5-8b6c-4a8b-a104-809f5b75549e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:50.484214 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1252/2523] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:50.485881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bec9eb-64b0-4778-851a-be9c1ad32881 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:50.502245 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bec9eb-64b0-4778-851a-be9c1ad32881 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:50.507134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd3e76-8be0-49c1-b10d-2f9481e1a677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:50.508073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd3e76-8be0-49c1-b10d-2f9481e1a677 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:50.508359 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd3e76-8be0-49c1-b10d-2f9481e1a677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:50.509333 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd3e76-8be0-49c1-b10d-2f9481e1a677 admin admin] PATH_INFO: `/v3/projects/cdd3f75984b349a0b33deb11fae51f27/users/2d7dfb44d22a42d4b6df435b03fe08a8/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:50.511402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd3e76-8be0-49c1-b10d-2f9481e1a677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdd3f75984b349a0b33deb11fae51f27, user_id=2d7dfb44d22a42d4b6df435b03fe08a8, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:50.514149 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd3e76-8be0-49c1-b10d-2f9481e1a677 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:50.531898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1272/2524] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:46:50 2022] PUT /identity/v3/projects/cdd3f75984b349a0b33deb11fae51f27/users/2d7dfb44d22a42d4b6df435b03fe08a8/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:50.547046 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83df5fd-8c41-424c-b5ce-8959f6e097d2 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:50.547349 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83df5fd-8c41-424c-b5ce-8959f6e097d2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:50.547629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83df5fd-8c41-424c-b5ce-8959f6e097d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:50.577152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a83df5fd-8c41-424c-b5ce-8959f6e097d2 None None] MFA Rules not processed for user `2d7dfb44d22a42d4b6df435b03fe08a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:50.578384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a83df5fd-8c41-424c-b5ce-8959f6e097d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:50.666628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1253/2525] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:50 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:50.681187 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd3e76-8be0-49c1-b10d-2f9481e1a677 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:50.682954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f5fd3e76-8be0-49c1-b10d-2f9481e1a677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:50.747739 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:50.754949 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd3e76-8be0-49c1-b10d-2f9481e1a677 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:50.761261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0854c-83be-40f3-b71c-7535850fb309 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:50.762526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0854c-83be-40f3-b71c-7535850fb309 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:50.762959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0854c-83be-40f3-b71c-7535850fb309 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:50.763458 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0854c-83be-40f3-b71c-7535850fb309 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:50.778790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f0854c-83be-40f3-b71c-7535850fb309 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:50.780997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f0854c-83be-40f3-b71c-7535850fb309 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:50.793237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1273/2526] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:50 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:50.880129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83df5fd-8c41-424c-b5ce-8959f6e097d2 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:50.890690 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83df5fd-8c41-424c-b5ce-8959f6e097d2 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:50.894360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94e274-2f94-4468-84e2-3f8a7ca95d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:50.895028 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94e274-2f94-4468-84e2-3f8a7ca95d8e service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:50.895328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94e274-2f94-4468-84e2-3f8a7ca95d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:50.895585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94e274-2f94-4468-84e2-3f8a7ca95d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:50.951084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:50.964605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94e274-2f94-4468-84e2-3f8a7ca95d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:50.968183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94e274-2f94-4468-84e2-3f8a7ca95d8e service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:50.978187 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1254/2527] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:51.435875 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f445a-739e-4f57-a401-4f5cc1195bc5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:51.436451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f445a-739e-4f57-a401-4f5cc1195bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:51.436451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f445a-739e-4f57-a401-4f5cc1195bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:51.455133 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-759f445a-739e-4f57-a401-4f5cc1195bc5 None None] MFA Rules not processed for user `d23a2825a06e4297a96ab098af99cd5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:51.456397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-759f445a-739e-4f57-a401-4f5cc1195bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:51.516067 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1274/2528] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:51.564767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94e274-2f94-4468-84e2-3f8a7ca95d8e service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:51.564767 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4e94e274-2f94-4468-84e2-3f8a7ca95d8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:51.571371 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f445a-739e-4f57-a401-4f5cc1195bc5 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:51.572961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-759f445a-739e-4f57-a401-4f5cc1195bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:51.632991 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94e274-2f94-4468-84e2-3f8a7ca95d8e service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:51.638180 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec40c18-660d-4a42-a45c-713f48b7de37 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e053b6b25a4e4a20a05e6ffa6683222b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:51.638962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec40c18-660d-4a42-a45c-713f48b7de37 service glance] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:51.638962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec40c18-660d-4a42-a45c-713f48b7de37 service glance] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:51.639186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec40c18-660d-4a42-a45c-713f48b7de37 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:51.641093 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2ec40c18-660d-4a42-a45c-713f48b7de37 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:51.641687 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f445a-739e-4f57-a401-4f5cc1195bc5 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:51.645572 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a1f53-94fd-4831-8d80-0e3353dd10a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:51.646237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a1f53-94fd-4831-8d80-0e3353dd10a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:51.646550 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a1f53-94fd-4831-8d80-0e3353dd10a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:51.646827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a1f53-94fd-4831-8d80-0e3353dd10a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:51.648684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-300a1f53-94fd-4831-8d80-0e3353dd10a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:51.718447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec40c18-660d-4a42-a45c-713f48b7de37 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:51.720124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec40c18-660d-4a42-a45c-713f48b7de37 service glance] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:51.720437 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a1f53-94fd-4831-8d80-0e3353dd10a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:51.721884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a1f53-94fd-4831-8d80-0e3353dd10a0 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:51.732405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1275/2529] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:46:51.732659 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1255/2530] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:52.004712 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a1f53-94fd-4831-8d80-0e3353dd10a0 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:52.006400 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-300a1f53-94fd-4831-8d80-0e3353dd10a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:52.019260 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a1f53-94fd-4831-8d80-0e3353dd10a0 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:52.024701 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528063fb-c644-4fda-81e2-0ab18144bb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:52.025597 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528063fb-c644-4fda-81e2-0ab18144bb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:52.025881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528063fb-c644-4fda-81e2-0ab18144bb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:52.026157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528063fb-c644-4fda-81e2-0ab18144bb1b admin admin] PATH_INFO: `/v3/users/b193b1915b1a42cbaf67e27144e4c060` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:52.027367 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528063fb-c644-4fda-81e2-0ab18144bb1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b193b1915b1a42cbaf67e27144e4c060)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:52.029978 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528063fb-c644-4fda-81e2-0ab18144bb1b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:52.076937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:52.085112 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-528063fb-c644-4fda-81e2-0ab18144bb1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b193b1915b1a42cbaf67e27144e4c060'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:52.085790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-528063fb-c644-4fda-81e2-0ab18144bb1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b193b1915b1a42cbaf67e27144e4c060'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:52.094731 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-528063fb-c644-4fda-81e2-0ab18144bb1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b193b1915b1a42cbaf67e27144e4c060'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:52.108675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-528063fb-c644-4fda-81e2-0ab18144bb1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b193b1915b1a42cbaf67e27144e4c060'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:52.116768 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-528063fb-c644-4fda-81e2-0ab18144bb1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b193b1915b1a42cbaf67e27144e4c060'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:52.124062 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1276/2531] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:46:52 2022] DELETE /identity/v3/users/b193b1915b1a42cbaf67e27144e4c060 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:52.130704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7102edb-c131-4643-9aa6-3a26590c9b76 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:52.131097 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7102edb-c131-4643-9aa6-3a26590c9b76 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:52.131460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7102edb-c131-4643-9aa6-3a26590c9b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:52.152322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e7102edb-c131-4643-9aa6-3a26590c9b76 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:52.154398 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e7102edb-c131-4643-9aa6-3a26590c9b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:52.230021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1256/2532] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:52.240693 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528063fb-c644-4fda-81e2-0ab18144bb1b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:52.242382 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-528063fb-c644-4fda-81e2-0ab18144bb1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:52.327873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528063fb-c644-4fda-81e2-0ab18144bb1b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:52.338686 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c61805-ba10-4349-b8f8-4b0566ecfd10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:52.338987 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c61805-ba10-4349-b8f8-4b0566ecfd10 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:52.339090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c61805-ba10-4349-b8f8-4b0566ecfd10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:52.339620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c61805-ba10-4349-b8f8-4b0566ecfd10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:52.351885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c61805-ba10-4349-b8f8-4b0566ecfd10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:52.354771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c61805-ba10-4349-b8f8-4b0566ecfd10 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:52.365585 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1277/2533] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:52.781808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7102edb-c131-4643-9aa6-3a26590c9b76 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:52.785332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e7102edb-c131-4643-9aa6-3a26590c9b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:52.814639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c61805-ba10-4349-b8f8-4b0566ecfd10 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:52.816410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7102edb-c131-4643-9aa6-3a26590c9b76 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:52.816553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e2c61805-ba10-4349-b8f8-4b0566ecfd10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:52.822081 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567a073-6468-4af0-b39f-137c515f8d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:52.822982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567a073-6468-4af0-b39f-137c515f8d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:52.823264 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567a073-6468-4af0-b39f-137c515f8d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:52.823648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567a073-6468-4af0-b39f-137c515f8d16 admin admin] PATH_INFO: `/v3/projects/96b471c181204a1582fe92816718094c` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:52.826037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567a073-6468-4af0-b39f-137c515f8d16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96b471c181204a1582fe92816718094c)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:52.828907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567a073-6468-4af0-b39f-137c515f8d16 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:52.865251 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:52.894860 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c567a073-6468-4af0-b39f-137c515f8d16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96b471c181204a1582fe92816718094c', 'request_id': 'req-c567a073-6468-4af0-b39f-137c515f8d16'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:52.901580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c567a073-6468-4af0-b39f-137c515f8d16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96b471c181204a1582fe92816718094c', 'request_id': 'req-c567a073-6468-4af0-b39f-137c515f8d16'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:52.911221 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c61805-ba10-4349-b8f8-4b0566ecfd10 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:52.911661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c567a073-6468-4af0-b39f-137c515f8d16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96b471c181204a1582fe92816718094c', 'request_id': 'req-c567a073-6468-4af0-b39f-137c515f8d16'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:52.925177 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c567a073-6468-4af0-b39f-137c515f8d16 admin admin] The token cache is being invalidate because project 96b471c181204a1582fe92816718094c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:46:52.925712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c567a073-6468-4af0-b39f-137c515f8d16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:52.926507 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c567a073-6468-4af0-b39f-137c515f8d16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:52.928397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1257/2534] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:46:52 2022] DELETE /identity/v3/projects/96b471c181204a1582fe92816718094c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:52.929296 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567a073-6468-4af0-b39f-137c515f8d16 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:52.931342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c567a073-6468-4af0-b39f-137c515f8d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:52.973419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:52.980967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d4784-1a79-4f46-aaf0-148ce0a36df9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:52.981921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d4784-1a79-4f46-aaf0-148ce0a36df9 service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:52.982103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d4784-1a79-4f46-aaf0-148ce0a36df9 service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:52.982267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d4784-1a79-4f46-aaf0-148ce0a36df9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:52.984784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-117d4784-1a79-4f46-aaf0-148ce0a36df9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:52.997502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d4784-1a79-4f46-aaf0-148ce0a36df9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:52.999180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d4784-1a79-4f46-aaf0-148ce0a36df9 service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:53.001002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567a073-6468-4af0-b39f-137c515f8d16 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:53.006321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147468f-eee9-4a77-a55e-d43679d04b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:53.007197 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147468f-eee9-4a77-a55e-d43679d04b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.007482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147468f-eee9-4a77-a55e-d43679d04b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.007793 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147468f-eee9-4a77-a55e-d43679d04b6f admin admin] PATH_INFO: `/v3/users/326bf42fe8c642b4898f2c8380733599` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.009045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147468f-eee9-4a77-a55e-d43679d04b6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=326bf42fe8c642b4898f2c8380733599)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:53.009573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1278/2535] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:46:53.010932 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00a6a4-f2ed-460a-8822-35a9637cc2ca None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.011222 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00a6a4-f2ed-460a-8822-35a9637cc2ca None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.011501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00a6a4-f2ed-460a-8822-35a9637cc2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.013541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147468f-eee9-4a77-a55e-d43679d04b6f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:53.037293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-df00a6a4-f2ed-460a-8822-35a9637cc2ca None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:53.038627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-df00a6a4-f2ed-460a-8822-35a9637cc2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:53.050040 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1279/2536] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:53.058181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799d873-3412-472c-8190-2c60c1d65d98 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.058586 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799d873-3412-472c-8190-2c60c1d65d98 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.058760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799d873-3412-472c-8190-2c60c1d65d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.067676 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3147468f-eee9-4a77-a55e-d43679d04b6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326bf42fe8c642b4898f2c8380733599'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:53.076885 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3147468f-eee9-4a77-a55e-d43679d04b6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326bf42fe8c642b4898f2c8380733599'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:53.079296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3799d873-3412-472c-8190-2c60c1d65d98 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:53.080553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3799d873-3412-472c-8190-2c60c1d65d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:53.084424 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3147468f-eee9-4a77-a55e-d43679d04b6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326bf42fe8c642b4898f2c8380733599'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:53.088584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1280/2537] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:46:53.094082 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799d873-3412-472c-8190-2c60c1d65d98 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:53.100746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3147468f-eee9-4a77-a55e-d43679d04b6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326bf42fe8c642b4898f2c8380733599'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:53.101481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3147468f-eee9-4a77-a55e-d43679d04b6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326bf42fe8c642b4898f2c8380733599'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:53.104032 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799d873-3412-472c-8190-2c60c1d65d98 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:53.108091 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1258/2538] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:46:52 2022] DELETE /identity/v3/users/326bf42fe8c642b4898f2c8380733599 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:53.109073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeb8d0-325b-42da-a2d2-9840adb719ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:53.109733 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeb8d0-325b-42da-a2d2-9840adb719ce admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.110050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeb8d0-325b-42da-a2d2-9840adb719ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.110330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeb8d0-325b-42da-a2d2-9840adb719ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.111236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efeb8d0-325b-42da-a2d2-9840adb719ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:46:53.111635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efeb8d0-325b-42da-a2d2-9840adb719ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:53.113212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efeb8d0-325b-42da-a2d2-9840adb719ce admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:53.129589 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1281/2539] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 09:46:53.130712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147468f-eee9-4a77-a55e-d43679d04b6f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:53.132284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3147468f-eee9-4a77-a55e-d43679d04b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:53.169667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:53.201756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147468f-eee9-4a77-a55e-d43679d04b6f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:53.206412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575aee90-37cb-4281-b677-a0092ce8cc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:53.207069 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575aee90-37cb-4281-b677-a0092ce8cc21 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.207355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575aee90-37cb-4281-b677-a0092ce8cc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.207632 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575aee90-37cb-4281-b677-a0092ce8cc21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.208381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575aee90-37cb-4281-b677-a0092ce8cc21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:53.210363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575aee90-37cb-4281-b677-a0092ce8cc21 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:53.237570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1259/2540] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:46:53 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:53.241587 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeb8d0-325b-42da-a2d2-9840adb719ce admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:53.251775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeb8d0-325b-42da-a2d2-9840adb719ce admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:53.296944 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575aee90-37cb-4281-b677-a0092ce8cc21 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:53.298865 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-575aee90-37cb-4281-b677-a0092ce8cc21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:53.315058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead02761-ef97-42c6-97d7-5c41d8901858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:53.315901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead02761-ef97-42c6-97d7-5c41d8901858 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.316235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead02761-ef97-42c6-97d7-5c41d8901858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.316577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead02761-ef97-42c6-97d7-5c41d8901858 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.317495 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead02761-ef97-42c6-97d7-5c41d8901858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:53.319558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead02761-ef97-42c6-97d7-5c41d8901858 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:53.348822 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1282/2541] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:46:53 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:53.353185 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead02761-ef97-42c6-97d7-5c41d8901858 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:53.364617 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead02761-ef97-42c6-97d7-5c41d8901858 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:53.369257 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575aee90-37cb-4281-b677-a0092ce8cc21 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:53.369397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c6460-7484-4f27-965e-833ec01a1fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:53.369930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c6460-7484-4f27-965e-833ec01a1fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.370216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c6460-7484-4f27-965e-833ec01a1fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.370544 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c6460-7484-4f27-965e-833ec01a1fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.371269 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c6460-7484-4f27-965e-833ec01a1fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:53.373709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c6460-7484-4f27-965e-833ec01a1fa2 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:53.374163 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ff066-dfbd-425c-9e15-1636b082223a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:53.375201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ff066-dfbd-425c-9e15-1636b082223a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.375489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ff066-dfbd-425c-9e15-1636b082223a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.376168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ff066-dfbd-425c-9e15-1636b082223a admin admin] PATH_INFO: `/v3/projects/afdbd56ebd1c48f1aa35d37819e000fa` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.377388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ff066-dfbd-425c-9e15-1636b082223a admin admin] RBAC: Authorizing `identity:delete_project(project_id=afdbd56ebd1c48f1aa35d37819e000fa)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:53.379863 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ff066-dfbd-425c-9e15-1636b082223a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:53.383952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1283/2542] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:46:53.388588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c6460-7484-4f27-965e-833ec01a1fa2 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:53.399550 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c6460-7484-4f27-965e-833ec01a1fa2 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:53.405331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0a16a-c054-4c12-ae32-9025919710d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:53.406005 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0a16a-c054-4c12-ae32-9025919710d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.406278 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0a16a-c054-4c12-ae32-9025919710d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.406680 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0a16a-c054-4c12-ae32-9025919710d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.407286 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0a16a-c054-4c12-ae32-9025919710d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:53.409136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0a16a-c054-4c12-ae32-9025919710d0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:53.421632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1284/2543] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:46:53.425830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0a16a-c054-4c12-ae32-9025919710d0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:53.429172 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ca7ff066-dfbd-425c-9e15-1636b082223a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afdbd56ebd1c48f1aa35d37819e000fa', 'request_id': 'req-ca7ff066-dfbd-425c-9e15-1636b082223a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:53.436168 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0a16a-c054-4c12-ae32-9025919710d0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:53.436412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ca7ff066-dfbd-425c-9e15-1636b082223a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afdbd56ebd1c48f1aa35d37819e000fa', 'request_id': 'req-ca7ff066-dfbd-425c-9e15-1636b082223a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:53.451001 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ca7ff066-dfbd-425c-9e15-1636b082223a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afdbd56ebd1c48f1aa35d37819e000fa', 'request_id': 'req-ca7ff066-dfbd-425c-9e15-1636b082223a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:53.461884 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ca7ff066-dfbd-425c-9e15-1636b082223a admin admin] The token cache is being invalidate because project afdbd56ebd1c48f1aa35d37819e000fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:46:53.462166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ca7ff066-dfbd-425c-9e15-1636b082223a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:53.462822 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ca7ff066-dfbd-425c-9e15-1636b082223a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:53.464474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1260/2544] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:46:53 2022] DELETE /identity/v3/projects/afdbd56ebd1c48f1aa35d37819e000fa => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:53.469529 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554314e9-d6d7-4bbf-851a-e6f06fc4c2dd None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.469825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554314e9-d6d7-4bbf-851a-e6f06fc4c2dd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.470105 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554314e9-d6d7-4bbf-851a-e6f06fc4c2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.488542 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-554314e9-d6d7-4bbf-851a-e6f06fc4c2dd None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:53.489681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-554314e9-d6d7-4bbf-851a-e6f06fc4c2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:53.496453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05ae45-a73e-463d-9e8b-ac402dcb5fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:53.497424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05ae45-a73e-463d-9e8b-ac402dcb5fdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.497706 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05ae45-a73e-463d-9e8b-ac402dcb5fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.497985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05ae45-a73e-463d-9e8b-ac402dcb5fdc admin admin] PATH_INFO: `/v3/projects/cd2483ea99d24899a70ff33da0236a21/users/8be69d6856df4c0090d2ad909826a8d6/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.513577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05ae45-a73e-463d-9e8b-ac402dcb5fdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd2483ea99d24899a70ff33da0236a21, user_id=8be69d6856df4c0090d2ad909826a8d6, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:53.516480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05ae45-a73e-463d-9e8b-ac402dcb5fdc admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:53.530088 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1285/2545] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:46:53 2022] PUT /identity/v3/projects/cd2483ea99d24899a70ff33da0236a21/users/8be69d6856df4c0090d2ad909826a8d6/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:46:53.536984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:53.557921 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1261/2546] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:46:53.566514 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d17bf-7267-40e5-bf4b-450fc513a034 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.566814 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d17bf-7267-40e5-bf4b-450fc513a034 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.567066 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d17bf-7267-40e5-bf4b-450fc513a034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.597247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-850d17bf-7267-40e5-bf4b-450fc513a034 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:53.597515 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487519f-619e-4dfb-a336-2c8874b1640f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.597789 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487519f-619e-4dfb-a336-2c8874b1640f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.598145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487519f-619e-4dfb-a336-2c8874b1640f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.598820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-850d17bf-7267-40e5-bf4b-450fc513a034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:53.609984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:53.628597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a487519f-619e-4dfb-a336-2c8874b1640f None None] MFA Rules not processed for user `8be69d6856df4c0090d2ad909826a8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:53.629963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a487519f-619e-4dfb-a336-2c8874b1640f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:53.671254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1286/2547] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:53.674532 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d17bf-7267-40e5-bf4b-450fc513a034 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:53.674532 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-850d17bf-7267-40e5-bf4b-450fc513a034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:53.706615 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1262/2548] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:46:53.707479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487519f-619e-4dfb-a336-2c8874b1640f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:53.725863 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487519f-619e-4dfb-a336-2c8874b1640f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:53.731966 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c0a75-6ed3-49f3-b918-db149e046db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:53.733746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c0a75-6ed3-49f3-b918-db149e046db9 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.734221 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c0a75-6ed3-49f3-b918-db149e046db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.734714 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c0a75-6ed3-49f3-b918-db149e046db9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.736190 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c0a75-6ed3-49f3-b918-db149e046db9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:46:53.736760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c0a75-6ed3-49f3-b918-db149e046db9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:53.738884 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c0a75-6ed3-49f3-b918-db149e046db9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:53.748793 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d17bf-7267-40e5-bf4b-450fc513a034 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:53.754356 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1263/2549] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:46:53.756842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c0a75-6ed3-49f3-b918-db149e046db9 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:53.758827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a08c0a75-6ed3-49f3-b918-db149e046db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:53.760018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639513d2-f439-4362-9724-13feec429f27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:53.761082 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639513d2-f439-4362-9724-13feec429f27 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.761527 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639513d2-f439-4362-9724-13feec429f27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.762057 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639513d2-f439-4362-9724-13feec429f27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.764697 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-639513d2-f439-4362-9724-13feec429f27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:53.830104 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c0a75-6ed3-49f3-b918-db149e046db9 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:53.834548 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817fff0-8aec-4152-9df6-4dda97d4c4b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:53.835207 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817fff0-8aec-4152-9df6-4dda97d4c4b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.835509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817fff0-8aec-4152-9df6-4dda97d4c4b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.835940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817fff0-8aec-4152-9df6-4dda97d4c4b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.846151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f817fff0-8aec-4152-9df6-4dda97d4c4b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:53.847633 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f817fff0-8aec-4152-9df6-4dda97d4c4b6 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:53.865165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1264/2550] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:53.866343 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817fff0-8aec-4152-9df6-4dda97d4c4b6 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:53.866835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639513d2-f439-4362-9724-13feec429f27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:53.868362 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639513d2-f439-4362-9724-13feec429f27 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:53.881280 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f817fff0-8aec-4152-9df6-4dda97d4c4b6 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:53.882348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1287/2551] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:53.893670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9033d-93ba-48b6-98c0-367f21a5fb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:53.894433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9033d-93ba-48b6-98c0-367f21a5fb1a admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:53.894820 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9033d-93ba-48b6-98c0-367f21a5fb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:53.895190 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9033d-93ba-48b6-98c0-367f21a5fb1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:53.896394 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9033d-93ba-48b6-98c0-367f21a5fb1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:53.898957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9033d-93ba-48b6-98c0-367f21a5fb1a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:53.945654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1265/2552] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:46:53 2022] POST /identity/v3/projects => generated 389 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:53.949887 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639513d2-f439-4362-9724-13feec429f27 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:53.962992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639513d2-f439-4362-9724-13feec429f27 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:54.037682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:54.046978 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a74a12-00eb-446f-b315-5dc9eb16c955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:54.047703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a74a12-00eb-446f-b315-5dc9eb16c955 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:54.048015 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a74a12-00eb-446f-b315-5dc9eb16c955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:54.048289 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a74a12-00eb-446f-b315-5dc9eb16c955 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:54.048982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a74a12-00eb-446f-b315-5dc9eb16c955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:54.051258 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a74a12-00eb-446f-b315-5dc9eb16c955 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:54.076111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1288/2553] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:46:53 2022] POST /identity/v3/users => generated 295 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:54.077449 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9033d-93ba-48b6-98c0-367f21a5fb1a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:54.087715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9033d-93ba-48b6-98c0-367f21a5fb1a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:54.092646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9fa01-8ef7-4284-9000-833410cf1338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:54.093290 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9fa01-8ef7-4284-9000-833410cf1338 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:54.093571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9fa01-8ef7-4284-9000-833410cf1338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:54.093845 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9fa01-8ef7-4284-9000-833410cf1338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:54.094566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9fa01-8ef7-4284-9000-833410cf1338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:54.096211 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9fa01-8ef7-4284-9000-833410cf1338 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:54.107016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1266/2554] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:46:54.110974 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a74a12-00eb-446f-b315-5dc9eb16c955 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:54.125563 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a74a12-00eb-446f-b315-5dc9eb16c955 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:54.130919 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2f00e-720e-4588-a957-d0a80f42262e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:54.131605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2f00e-720e-4588-a957-d0a80f42262e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:54.131874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2f00e-720e-4588-a957-d0a80f42262e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:54.132190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2f00e-720e-4588-a957-d0a80f42262e admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:54.132924 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2f00e-720e-4588-a957-d0a80f42262e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:54.134734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2f00e-720e-4588-a957-d0a80f42262e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:54.146120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1289/2555] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:46:54.150260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9fa01-8ef7-4284-9000-833410cf1338 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:54.161431 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9fa01-8ef7-4284-9000-833410cf1338 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:54.171521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ed840-a2bc-4ff4-943b-9a020ac96e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:54.172612 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ed840-a2bc-4ff4-943b-9a020ac96e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:54.172968 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ed840-a2bc-4ff4-943b-9a020ac96e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:54.173320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ed840-a2bc-4ff4-943b-9a020ac96e57 admin admin] PATH_INFO: `/v3/projects/6f07628e8d854e99a072e66429d1b950/users/e3ae8bc60a7449cea3da7e9f0e063f77/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:54.192163 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ed840-a2bc-4ff4-943b-9a020ac96e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f07628e8d854e99a072e66429d1b950, user_id=e3ae8bc60a7449cea3da7e9f0e063f77, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:54.195789 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ed840-a2bc-4ff4-943b-9a020ac96e57 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:54.214807 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1267/2556] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:46:54 2022] PUT /identity/v3/projects/6f07628e8d854e99a072e66429d1b950/users/e3ae8bc60a7449cea3da7e9f0e063f77/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:46:54.226729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08a089-8a30-4ecd-8335-8c43024eb212 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:54.228491 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08a089-8a30-4ecd-8335-8c43024eb212 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:54.228491 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08a089-8a30-4ecd-8335-8c43024eb212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:54.259712 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-fd08a089-8a30-4ecd-8335-8c43024eb212 None None] MFA Rules not processed for user `e3ae8bc60a7449cea3da7e9f0e063f77`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:54.260993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fd08a089-8a30-4ecd-8335-8c43024eb212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:54.331179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1290/2557] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:54.343476 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ed840-a2bc-4ff4-943b-9a020ac96e57 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:54.356661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:54.400825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ed840-a2bc-4ff4-943b-9a020ac96e57 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:54.405291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd9961-98ee-43d5-8dce-5163f8217f39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:54.405926 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd9961-98ee-43d5-8dce-5163f8217f39 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:54.406210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd9961-98ee-43d5-8dce-5163f8217f39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:54.406485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd9961-98ee-43d5-8dce-5163f8217f39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:54.415018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd9961-98ee-43d5-8dce-5163f8217f39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:54.416492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd9961-98ee-43d5-8dce-5163f8217f39 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:54.425599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1268/2558] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:55.203083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08a089-8a30-4ecd-8335-8c43024eb212 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:55.204723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fd08a089-8a30-4ecd-8335-8c43024eb212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:55.274082 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08a089-8a30-4ecd-8335-8c43024eb212 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:55.278916 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad06b9-0571-4b18-b1d9-0fd63ac93824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:55.279780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad06b9-0571-4b18-b1d9-0fd63ac93824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:55.280109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad06b9-0571-4b18-b1d9-0fd63ac93824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:55.280434 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad06b9-0571-4b18-b1d9-0fd63ac93824 admin admin] PATH_INFO: `/v3/users/d200fb43b3c448d49e85e03f5eaab88e` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:55.281657 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad06b9-0571-4b18-b1d9-0fd63ac93824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d200fb43b3c448d49e85e03f5eaab88e)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:55.283588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad06b9-0571-4b18-b1d9-0fd63ac93824 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:55.342969 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a4ad06b9-0571-4b18-b1d9-0fd63ac93824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd200fb43b3c448d49e85e03f5eaab88e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:55.343676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a4ad06b9-0571-4b18-b1d9-0fd63ac93824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd200fb43b3c448d49e85e03f5eaab88e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:55.352070 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a4ad06b9-0571-4b18-b1d9-0fd63ac93824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd200fb43b3c448d49e85e03f5eaab88e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:55.363295 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:55.365674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a4ad06b9-0571-4b18-b1d9-0fd63ac93824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd200fb43b3c448d49e85e03f5eaab88e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:55.375539 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a4ad06b9-0571-4b18-b1d9-0fd63ac93824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd200fb43b3c448d49e85e03f5eaab88e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:55.381933 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1291/2559] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:46:55 2022] DELETE /identity/v3/users/d200fb43b3c448d49e85e03f5eaab88e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:55.386665 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57af2cb-781f-4bd1-95da-87c8e3987ea2 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:55.387147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57af2cb-781f-4bd1-95da-87c8e3987ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:55.387455 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57af2cb-781f-4bd1-95da-87c8e3987ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:55.408712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a57af2cb-781f-4bd1-95da-87c8e3987ea2 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:55.409917 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a57af2cb-781f-4bd1-95da-87c8e3987ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:55.476644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1269/2560] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:55.487047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad06b9-0571-4b18-b1d9-0fd63ac93824 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:55.488479 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a4ad06b9-0571-4b18-b1d9-0fd63ac93824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:55.554922 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad06b9-0571-4b18-b1d9-0fd63ac93824 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:55.558871 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ab9b7-cfc6-4e53-a1ba-7af428148627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:55.559559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ab9b7-cfc6-4e53-a1ba-7af428148627 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:55.559843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ab9b7-cfc6-4e53-a1ba-7af428148627 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:55.560290 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ab9b7-cfc6-4e53-a1ba-7af428148627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:55.569654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ab9b7-cfc6-4e53-a1ba-7af428148627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:55.571096 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ab9b7-cfc6-4e53-a1ba-7af428148627 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:55.582090 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1292/2561] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:46:55.919936 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57af2cb-781f-4bd1-95da-87c8e3987ea2 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:55.921531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a57af2cb-781f-4bd1-95da-87c8e3987ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:55.938473 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57af2cb-781f-4bd1-95da-87c8e3987ea2 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:55.947885 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf135c6d-932f-4968-b460-27aaef338864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:55.949042 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf135c6d-932f-4968-b460-27aaef338864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:55.949412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf135c6d-932f-4968-b460-27aaef338864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:55.950281 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf135c6d-932f-4968-b460-27aaef338864 admin admin] PATH_INFO: `/v3/projects/5cf8330933a3474abcfdce0d65186f2e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:55.952889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf135c6d-932f-4968-b460-27aaef338864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cf8330933a3474abcfdce0d65186f2e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:55.955032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf135c6d-932f-4968-b460-27aaef338864 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:56.015073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cf135c6d-932f-4968-b460-27aaef338864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cf8330933a3474abcfdce0d65186f2e', 'request_id': 'req-cf135c6d-932f-4968-b460-27aaef338864'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:56.021810 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cf135c6d-932f-4968-b460-27aaef338864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cf8330933a3474abcfdce0d65186f2e', 'request_id': 'req-cf135c6d-932f-4968-b460-27aaef338864'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:56.032298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cf135c6d-932f-4968-b460-27aaef338864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cf8330933a3474abcfdce0d65186f2e', 'request_id': 'req-cf135c6d-932f-4968-b460-27aaef338864'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:56.047113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cf135c6d-932f-4968-b460-27aaef338864 admin admin] The token cache is being invalidate because project 5cf8330933a3474abcfdce0d65186f2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:46:56.047401 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cf135c6d-932f-4968-b460-27aaef338864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:56.048147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cf135c6d-932f-4968-b460-27aaef338864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:56.049740 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1270/2562] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:46:55 2022] DELETE /identity/v3/projects/5cf8330933a3474abcfdce0d65186f2e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:56.053896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ab9b7-cfc6-4e53-a1ba-7af428148627 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:56.064352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ab9b7-cfc6-4e53-a1ba-7af428148627 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:56.135057 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6b1c5-4756-49fb-9369-2bc3df8c8929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:56.135943 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6b1c5-4756-49fb-9369-2bc3df8c8929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:56.136235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6b1c5-4756-49fb-9369-2bc3df8c8929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:56.136515 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6b1c5-4756-49fb-9369-2bc3df8c8929 admin admin] PATH_INFO: `/v3/users/2d7dfb44d22a42d4b6df435b03fe08a8` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:56.138360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6b1c5-4756-49fb-9369-2bc3df8c8929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d7dfb44d22a42d4b6df435b03fe08a8)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:56.140350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6b1c5-4756-49fb-9369-2bc3df8c8929 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:56.188815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dac6b1c5-4756-49fb-9369-2bc3df8c8929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d7dfb44d22a42d4b6df435b03fe08a8'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:56.189478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dac6b1c5-4756-49fb-9369-2bc3df8c8929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d7dfb44d22a42d4b6df435b03fe08a8'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:56.197376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dac6b1c5-4756-49fb-9369-2bc3df8c8929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d7dfb44d22a42d4b6df435b03fe08a8'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:56.210650 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dac6b1c5-4756-49fb-9369-2bc3df8c8929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d7dfb44d22a42d4b6df435b03fe08a8'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:56.218134 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dac6b1c5-4756-49fb-9369-2bc3df8c8929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d7dfb44d22a42d4b6df435b03fe08a8'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:56.226056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1293/2563] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:46:56 2022] DELETE /identity/v3/users/2d7dfb44d22a42d4b6df435b03fe08a8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:46:56.536442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf135c6d-932f-4968-b460-27aaef338864 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:56.537992 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-cf135c6d-932f-4968-b460-27aaef338864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:56.556418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf135c6d-932f-4968-b460-27aaef338864 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:56.590762 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:56.615874 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4750b-3b1f-4da8-96fd-b7a348d60595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:56.616765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4750b-3b1f-4da8-96fd-b7a348d60595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:56.617063 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4750b-3b1f-4da8-96fd-b7a348d60595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:56.617340 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4750b-3b1f-4da8-96fd-b7a348d60595 admin admin] PATH_INFO: `/v3/projects/cdd3f75984b349a0b33deb11fae51f27` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:56.618528 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4750b-3b1f-4da8-96fd-b7a348d60595 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdd3f75984b349a0b33deb11fae51f27)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:56.620569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4750b-3b1f-4da8-96fd-b7a348d60595 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:56.670897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dcb4750b-3b1f-4da8-96fd-b7a348d60595 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdd3f75984b349a0b33deb11fae51f27', 'request_id': 'req-dcb4750b-3b1f-4da8-96fd-b7a348d60595'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:56.677913 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dcb4750b-3b1f-4da8-96fd-b7a348d60595 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdd3f75984b349a0b33deb11fae51f27', 'request_id': 'req-dcb4750b-3b1f-4da8-96fd-b7a348d60595'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:56.686674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dcb4750b-3b1f-4da8-96fd-b7a348d60595 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdd3f75984b349a0b33deb11fae51f27', 'request_id': 'req-dcb4750b-3b1f-4da8-96fd-b7a348d60595'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:56.698140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dcb4750b-3b1f-4da8-96fd-b7a348d60595 admin admin] The token cache is being invalidate because project cdd3f75984b349a0b33deb11fae51f27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:46:56.698424 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dcb4750b-3b1f-4da8-96fd-b7a348d60595 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:56.699082 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dcb4750b-3b1f-4da8-96fd-b7a348d60595 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:56.701334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1271/2564] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:46:56 2022] DELETE /identity/v3/projects/cdd3f75984b349a0b33deb11fae51f27 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:56.706274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccc687-6fbb-47b6-9768-b36cd0deef86 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:56.706618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccc687-6fbb-47b6-9768-b36cd0deef86 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:56.706900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccc687-6fbb-47b6-9768-b36cd0deef86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:56.724804 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:56.732781 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f7ccc687-6fbb-47b6-9768-b36cd0deef86 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:56.734626 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f7ccc687-6fbb-47b6-9768-b36cd0deef86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:56.803243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1294/2565] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:46:56.811665 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c62db3-9d13-49fe-b542-11df194b657f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:56.811792 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c62db3-9d13-49fe-b542-11df194b657f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:56.812068 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c62db3-9d13-49fe-b542-11df194b657f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:56.830811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-71c62db3-9d13-49fe-b542-11df194b657f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:56.832080 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-71c62db3-9d13-49fe-b542-11df194b657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:56.839990 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1272/2566] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:56.844687 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccc687-6fbb-47b6-9768-b36cd0deef86 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:56.855306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ccc687-6fbb-47b6-9768-b36cd0deef86 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:56.861569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6b5fc-d9d3-4321-8a90-f04969622bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:56.862226 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6b5fc-d9d3-4321-8a90-f04969622bcf admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:56.862514 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6b5fc-d9d3-4321-8a90-f04969622bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:56.862786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6b5fc-d9d3-4321-8a90-f04969622bcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:56.863516 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6b5fc-d9d3-4321-8a90-f04969622bcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:46:56.863819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6b5fc-d9d3-4321-8a90-f04969622bcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:56.865457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6b5fc-d9d3-4321-8a90-f04969622bcf admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:56.879111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1295/2567] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:46:56.883702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c62db3-9d13-49fe-b542-11df194b657f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:56.901545 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c62db3-9d13-49fe-b542-11df194b657f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:56.908577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec25364-b1c4-4605-b8a1-2b9011287181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:56.909451 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec25364-b1c4-4605-b8a1-2b9011287181 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:56.909881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec25364-b1c4-4605-b8a1-2b9011287181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:56.910200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec25364-b1c4-4605-b8a1-2b9011287181 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:56.911136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec25364-b1c4-4605-b8a1-2b9011287181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:56.913525 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec25364-b1c4-4605-b8a1-2b9011287181 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:56.945642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1273/2568] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:46:56 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:56.950561 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6b5fc-d9d3-4321-8a90-f04969622bcf admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:56.963899 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6b5fc-d9d3-4321-8a90-f04969622bcf admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:57.002097 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec25364-b1c4-4605-b8a1-2b9011287181 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:57.003883 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2ec25364-b1c4-4605-b8a1-2b9011287181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:57.058720 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009d0d0-da07-445e-9548-896bf89c0f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:57.059481 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009d0d0-da07-445e-9548-896bf89c0f19 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.059803 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009d0d0-da07-445e-9548-896bf89c0f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.060671 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009d0d0-da07-445e-9548-896bf89c0f19 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.061520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009d0d0-da07-445e-9548-896bf89c0f19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:57.063894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009d0d0-da07-445e-9548-896bf89c0f19 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:57.089922 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1296/2569] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:46:56 2022] POST /identity/v3/users => generated 284 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:46:57.091857 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009d0d0-da07-445e-9548-896bf89c0f19 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:57.096556 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec25364-b1c4-4605-b8a1-2b9011287181 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:57.102502 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009d0d0-da07-445e-9548-896bf89c0f19 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:57.104867 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095a1fa-bbb4-444a-b220-eb8322d3b11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:57.106738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095a1fa-bbb4-444a-b220-eb8322d3b11f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.107325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095a1fa-bbb4-444a-b220-eb8322d3b11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.107685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095a1fa-bbb4-444a-b220-eb8322d3b11f admin admin] PATH_INFO: `/v3/users/8be69d6856df4c0090d2ad909826a8d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.109115 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a07506-a71d-4492-b12c-2628127166af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:57.109298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095a1fa-bbb4-444a-b220-eb8322d3b11f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8be69d6856df4c0090d2ad909826a8d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:57.109734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a07506-a71d-4492-b12c-2628127166af admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.110043 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a07506-a71d-4492-b12c-2628127166af admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.111675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a07506-a71d-4492-b12c-2628127166af admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.111675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a07506-a71d-4492-b12c-2628127166af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:57.112054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095a1fa-bbb4-444a-b220-eb8322d3b11f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:57.113229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a07506-a71d-4492-b12c-2628127166af admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:57.128894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1297/2570] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:57.131202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a07506-a71d-4492-b12c-2628127166af admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:57.143239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a07506-a71d-4492-b12c-2628127166af admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:57.149435 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07172c-c3c6-4c31-a825-690db2c10375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:57.150213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07172c-c3c6-4c31-a825-690db2c10375 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.150553 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07172c-c3c6-4c31-a825-690db2c10375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.150893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07172c-c3c6-4c31-a825-690db2c10375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.151770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07172c-c3c6-4c31-a825-690db2c10375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:57.154264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07172c-c3c6-4c31-a825-690db2c10375 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:57.165164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:57.168401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1298/2571] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:57.172440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e095a1fa-bbb4-444a-b220-eb8322d3b11f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8be69d6856df4c0090d2ad909826a8d6'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:57.173530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07172c-c3c6-4c31-a825-690db2c10375 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:57.182606 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e095a1fa-bbb4-444a-b220-eb8322d3b11f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8be69d6856df4c0090d2ad909826a8d6'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:57.187527 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07172c-c3c6-4c31-a825-690db2c10375 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:57.191280 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e095a1fa-bbb4-444a-b220-eb8322d3b11f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8be69d6856df4c0090d2ad909826a8d6'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:57.193509 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5136101-02c8-46e4-859c-cb91b3315049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:57.194401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5136101-02c8-46e4-859c-cb91b3315049 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.194701 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5136101-02c8-46e4-859c-cb91b3315049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.194982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5136101-02c8-46e4-859c-cb91b3315049 admin admin] PATH_INFO: `/v3/projects/223f82649e1e485da3fdbc8dabf96275/users/7d5083cc13fa4909bf5b4919aea5aede/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.206126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e095a1fa-bbb4-444a-b220-eb8322d3b11f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8be69d6856df4c0090d2ad909826a8d6'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:57.206884 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e095a1fa-bbb4-444a-b220-eb8322d3b11f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8be69d6856df4c0090d2ad909826a8d6'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:57.211895 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5136101-02c8-46e4-859c-cb91b3315049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=223f82649e1e485da3fdbc8dabf96275, user_id=7d5083cc13fa4909bf5b4919aea5aede, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:57.216757 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1274/2572] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:46:57 2022] DELETE /identity/v3/users/8be69d6856df4c0090d2ad909826a8d6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:46:57.217588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5136101-02c8-46e4-859c-cb91b3315049 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:57.218121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0641c-8008-4fe7-b16a-1a1d9b49ece1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.218410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0641c-8008-4fe7-b16a-1a1d9b49ece1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.218687 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0641c-8008-4fe7-b16a-1a1d9b49ece1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.232184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1299/2573] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:46:57 2022] PUT /identity/v3/projects/223f82649e1e485da3fdbc8dabf96275/users/7d5083cc13fa4909bf5b4919aea5aede/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:46:57.240742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c9f0641c-8008-4fe7-b16a-1a1d9b49ece1 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:57.242042 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c9f0641c-8008-4fe7-b16a-1a1d9b49ece1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:57.242552 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efff4db-498e-4391-9062-9a98f22292e1 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.242848 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efff4db-498e-4391-9062-9a98f22292e1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.243122 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efff4db-498e-4391-9062-9a98f22292e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.270490 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6efff4db-498e-4391-9062-9a98f22292e1 None None] MFA Rules not processed for user `7d5083cc13fa4909bf5b4919aea5aede`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:57.271750 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6efff4db-498e-4391-9062-9a98f22292e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:57.298280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:57.304156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1275/2574] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:46:57.314043 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0641c-8008-4fe7-b16a-1a1d9b49ece1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:57.315444 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c9f0641c-8008-4fe7-b16a-1a1d9b49ece1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:57.340564 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1300/2575] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:46:57.345126 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efff4db-498e-4391-9062-9a98f22292e1 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:57.346441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6efff4db-498e-4391-9062-9a98f22292e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:57.361175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efff4db-498e-4391-9062-9a98f22292e1 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:57.365961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc435c-4740-4c85-871b-b662e05041ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:57.366627 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc435c-4740-4c85-871b-b662e05041ab admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.366912 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc435c-4740-4c85-871b-b662e05041ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.367191 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc435c-4740-4c85-871b-b662e05041ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.367899 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc435c-4740-4c85-871b-b662e05041ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:57.369962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc435c-4740-4c85-871b-b662e05041ab admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:57.381434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0641c-8008-4fe7-b16a-1a1d9b49ece1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:57.385218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7508eb-b652-4e83-9518-2eeb7957b329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:57.385878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7508eb-b652-4e83-9518-2eeb7957b329 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.386162 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7508eb-b652-4e83-9518-2eeb7957b329 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.386442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7508eb-b652-4e83-9518-2eeb7957b329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.397116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7508eb-b652-4e83-9518-2eeb7957b329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:57.398377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1301/2576] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:46:57 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:46:57.398794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7508eb-b652-4e83-9518-2eeb7957b329 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:57.402365 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc435c-4740-4c85-871b-b662e05041ab admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:57.409652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1276/2577] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:46:57.486156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc435c-4740-4c85-871b-b662e05041ab admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:57.492226 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2ba29-4354-42f7-8e70-a3ef404e505d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:57.492908 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2ba29-4354-42f7-8e70-a3ef404e505d admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.493215 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2ba29-4354-42f7-8e70-a3ef404e505d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.493457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2ba29-4354-42f7-8e70-a3ef404e505d admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.494163 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2ba29-4354-42f7-8e70-a3ef404e505d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:57.496137 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2ba29-4354-42f7-8e70-a3ef404e505d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:57.521988 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1302/2578] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:46:57 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:46:57.523869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7508eb-b652-4e83-9518-2eeb7957b329 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:57.534823 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7508eb-b652-4e83-9518-2eeb7957b329 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:57.540677 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad0114-da60-47e2-bbbb-f7a965ef8ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:57.542108 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad0114-da60-47e2-bbbb-f7a965ef8ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.542246 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad0114-da60-47e2-bbbb-f7a965ef8ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.542599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad0114-da60-47e2-bbbb-f7a965ef8ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.544566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad0114-da60-47e2-bbbb-f7a965ef8ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:57.546481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad0114-da60-47e2-bbbb-f7a965ef8ba2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:57.557040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1277/2579] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:57.562024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2ba29-4354-42f7-8e70-a3ef404e505d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:57.572907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2ba29-4354-42f7-8e70-a3ef404e505d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:57.579257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8efef-1290-4e9e-883a-5bb578ede868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:57.580856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8efef-1290-4e9e-883a-5bb578ede868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.581030 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8efef-1290-4e9e-883a-5bb578ede868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.581729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8efef-1290-4e9e-883a-5bb578ede868 admin admin] PATH_INFO: `/v3/projects/4ea30662d0af420bb9688ea9b7e6192c/users/17bdd737b4c942eeb42475a59f474cdb/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.598945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8efef-1290-4e9e-883a-5bb578ede868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea30662d0af420bb9688ea9b7e6192c, user_id=17bdd737b4c942eeb42475a59f474cdb, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:57.601601 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8efef-1290-4e9e-883a-5bb578ede868 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:57.617199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1303/2580] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:46:57 2022] PUT /identity/v3/projects/4ea30662d0af420bb9688ea9b7e6192c/users/17bdd737b4c942eeb42475a59f474cdb/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:46:57.621753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad0114-da60-47e2-bbbb-f7a965ef8ba2 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:57.637796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad0114-da60-47e2-bbbb-f7a965ef8ba2 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:57.686897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:57.699158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc9fec-58ad-4177-bea7-15c0ac6c6f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:57.699158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc9fec-58ad-4177-bea7-15c0ac6c6f56 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.699495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc9fec-58ad-4177-bea7-15c0ac6c6f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.699618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc9fec-58ad-4177-bea7-15c0ac6c6f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.701011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc9fec-58ad-4177-bea7-15c0ac6c6f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:57.702905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc9fec-58ad-4177-bea7-15c0ac6c6f56 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:57.718531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1278/2581] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:57.720602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8efef-1290-4e9e-883a-5bb578ede868 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:57.725608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc9fec-58ad-4177-bea7-15c0ac6c6f56 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:57.728010 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6fdc9fec-58ad-4177-bea7-15c0ac6c6f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:57.739695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8efef-1290-4e9e-883a-5bb578ede868 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:57.744833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ef624-27f2-4748-93c4-9a1b44093cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:57.745740 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ef624-27f2-4748-93c4-9a1b44093cad admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.746024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ef624-27f2-4748-93c4-9a1b44093cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.746300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ef624-27f2-4748-93c4-9a1b44093cad admin admin] PATH_INFO: `/v3/projects/4ea30662d0af420bb9688ea9b7e6192c/users/17bdd737b4c942eeb42475a59f474cdb/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.746787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc9fec-58ad-4177-bea7-15c0ac6c6f56 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:57.748683 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ef624-27f2-4748-93c4-9a1b44093cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea30662d0af420bb9688ea9b7e6192c, user_id=17bdd737b4c942eeb42475a59f474cdb, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:57.751412 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ef624-27f2-4748-93c4-9a1b44093cad admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:57.752432 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283bacf-5deb-4c8f-94e5-30796aa22408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:57.753298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283bacf-5deb-4c8f-94e5-30796aa22408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.754332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283bacf-5deb-4c8f-94e5-30796aa22408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.754625 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283bacf-5deb-4c8f-94e5-30796aa22408 admin admin] PATH_INFO: `/v3/projects/cd2483ea99d24899a70ff33da0236a21` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.756491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283bacf-5deb-4c8f-94e5-30796aa22408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd2483ea99d24899a70ff33da0236a21)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:57.758618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283bacf-5deb-4c8f-94e5-30796aa22408 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:57.763692 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1304/2582] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:46:57 2022] PUT /identity/v3/projects/4ea30662d0af420bb9688ea9b7e6192c/users/17bdd737b4c942eeb42475a59f474cdb/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:57.815357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c283bacf-5deb-4c8f-94e5-30796aa22408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd2483ea99d24899a70ff33da0236a21', 'request_id': 'req-c283bacf-5deb-4c8f-94e5-30796aa22408'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:57.823458 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c283bacf-5deb-4c8f-94e5-30796aa22408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd2483ea99d24899a70ff33da0236a21', 'request_id': 'req-c283bacf-5deb-4c8f-94e5-30796aa22408'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:57.829866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf74cd-d3bd-4221-806d-77d920086725 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.830180 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf74cd-d3bd-4221-806d-77d920086725 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.830431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf74cd-d3bd-4221-806d-77d920086725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.833056 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c283bacf-5deb-4c8f-94e5-30796aa22408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd2483ea99d24899a70ff33da0236a21', 'request_id': 'req-c283bacf-5deb-4c8f-94e5-30796aa22408'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:57.844820 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c283bacf-5deb-4c8f-94e5-30796aa22408 admin admin] The token cache is being invalidate because project cd2483ea99d24899a70ff33da0236a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:46:57.845166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c283bacf-5deb-4c8f-94e5-30796aa22408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:57.845854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c283bacf-5deb-4c8f-94e5-30796aa22408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:46:57.847855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1279/2583] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:46:57 2022] DELETE /identity/v3/projects/cd2483ea99d24899a70ff33da0236a21 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:46:57.853472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced539bf-d702-4f4c-9bab-8ca9e0e34818 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.853642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced539bf-d702-4f4c-9bab-8ca9e0e34818 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.854034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced539bf-d702-4f4c-9bab-8ca9e0e34818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.860216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f4bf74cd-d3bd-4221-806d-77d920086725 None None] MFA Rules not processed for user `17bdd737b4c942eeb42475a59f474cdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:57.861550 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f4bf74cd-d3bd-4221-806d-77d920086725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:57.872814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ced539bf-d702-4f4c-9bab-8ca9e0e34818 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:57.873999 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ced539bf-d702-4f4c-9bab-8ca9e0e34818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:57.911613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:57.937629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1280/2584] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:57.938018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1305/2585] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:57.945250 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2edc2-0ec5-4b63-9201-4be8b3f14685 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.945629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2edc2-0ec5-4b63-9201-4be8b3f14685 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.945984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2edc2-0ec5-4b63-9201-4be8b3f14685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.955325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf74cd-d3bd-4221-806d-77d920086725 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:57.956747 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f4bf74cd-d3bd-4221-806d-77d920086725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:57.964843 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-eda2edc2-0ec5-4b63-9201-4be8b3f14685 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:57.966562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-eda2edc2-0ec5-4b63-9201-4be8b3f14685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:57.974589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1281/2586] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:46:57.978699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2edc2-0ec5-4b63-9201-4be8b3f14685 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:57.987753 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2edc2-0ec5-4b63-9201-4be8b3f14685 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:57.992462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2b6fd-cd5f-477d-b490-3f1b26cb5126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:57.993106 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2b6fd-cd5f-477d-b490-3f1b26cb5126 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:57.993389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2b6fd-cd5f-477d-b490-3f1b26cb5126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:57.993668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2b6fd-cd5f-477d-b490-3f1b26cb5126 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:57.994391 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2b6fd-cd5f-477d-b490-3f1b26cb5126 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:46:57.994699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2b6fd-cd5f-477d-b490-3f1b26cb5126 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:57.996383 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2b6fd-cd5f-477d-b490-3f1b26cb5126 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:58.008934 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1282/2587] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 09:46:58.013908 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2b6fd-cd5f-477d-b490-3f1b26cb5126 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:58.017056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf74cd-d3bd-4221-806d-77d920086725 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:58.021029 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc169b6a-4f18-4c58-9ee6-237b75afdfda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:58.021680 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc169b6a-4f18-4c58-9ee6-237b75afdfda service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:58.021958 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc169b6a-4f18-4c58-9ee6-237b75afdfda service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:58.022228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc169b6a-4f18-4c58-9ee6-237b75afdfda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:58.023686 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2b6fd-cd5f-477d-b490-3f1b26cb5126 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:58.028889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052b1e7-2e6a-4c17-8cfb-5fa346b48ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:58.029528 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052b1e7-2e6a-4c17-8cfb-5fa346b48ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:58.029817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052b1e7-2e6a-4c17-8cfb-5fa346b48ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:58.030109 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052b1e7-2e6a-4c17-8cfb-5fa346b48ca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:58.030795 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052b1e7-2e6a-4c17-8cfb-5fa346b48ca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:58.032559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc169b6a-4f18-4c58-9ee6-237b75afdfda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:58.032756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052b1e7-2e6a-4c17-8cfb-5fa346b48ca9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:58.033976 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc169b6a-4f18-4c58-9ee6-237b75afdfda service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:58.045325 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1306/2588] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:46:58.064616 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1283/2589] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:46:58 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:46:58.068884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc169b6a-4f18-4c58-9ee6-237b75afdfda service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:58.080958 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc169b6a-4f18-4c58-9ee6-237b75afdfda service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:58.133000 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052b1e7-2e6a-4c17-8cfb-5fa346b48ca9 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:58.145969 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a636a7-1b91-46a9-960a-eaf2ef8c9081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:58.146672 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a636a7-1b91-46a9-960a-eaf2ef8c9081 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:58.146954 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a636a7-1b91-46a9-960a-eaf2ef8c9081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:58.147235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a636a7-1b91-46a9-960a-eaf2ef8c9081 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:58.147966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a636a7-1b91-46a9-960a-eaf2ef8c9081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:58.149945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a636a7-1b91-46a9-960a-eaf2ef8c9081 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:58.167267 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:58.175818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1307/2590] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:46:58 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:46:58.176872 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a636a7-1b91-46a9-960a-eaf2ef8c9081 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:58.187342 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a636a7-1b91-46a9-960a-eaf2ef8c9081 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:58.192013 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0933832-8a9a-4e38-9833-17307991edd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:58.192676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0933832-8a9a-4e38-9833-17307991edd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:58.192963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0933832-8a9a-4e38-9833-17307991edd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:58.193273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0933832-8a9a-4e38-9833-17307991edd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:58.194000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0933832-8a9a-4e38-9833-17307991edd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:58.195496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052b1e7-2e6a-4c17-8cfb-5fa346b48ca9 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:58.195665 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0933832-8a9a-4e38-9833-17307991edd2 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:58.199492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc88e9-a2d6-4709-8f84-c01cad304fbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:58.200902 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc88e9-a2d6-4709-8f84-c01cad304fbf service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:58.201031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc88e9-a2d6-4709-8f84-c01cad304fbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:58.201289 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc88e9-a2d6-4709-8f84-c01cad304fbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:58.203122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-03bc88e9-a2d6-4709-8f84-c01cad304fbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:58.209681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1308/2591] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:46:58.210533 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0933832-8a9a-4e38-9833-17307991edd2 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:58.221226 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0933832-8a9a-4e38-9833-17307991edd2 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:58.228080 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96919db-789d-4ed2-94b8-1746ad6403a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:58.228295 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96919db-789d-4ed2-94b8-1746ad6403a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:58.228527 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96919db-789d-4ed2-94b8-1746ad6403a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:58.229095 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96919db-789d-4ed2-94b8-1746ad6403a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:58.229830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96919db-789d-4ed2-94b8-1746ad6403a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:58.231548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96919db-789d-4ed2-94b8-1746ad6403a7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:58.243707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1309/2592] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:46:58.248094 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96919db-789d-4ed2-94b8-1746ad6403a7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:58.258174 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96919db-789d-4ed2-94b8-1746ad6403a7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:58.263431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04872c1-1f48-4806-9735-459e60ea1a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:58.264347 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04872c1-1f48-4806-9735-459e60ea1a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:58.264626 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04872c1-1f48-4806-9735-459e60ea1a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:58.264924 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04872c1-1f48-4806-9735-459e60ea1a5a admin admin] PATH_INFO: `/v3/projects/0ff87f1adad5487ab49f7f4ae8859df7/users/6dd9a208282d4a35a444069855b8b1d5/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:58.276565 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc88e9-a2d6-4709-8f84-c01cad304fbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:58.278676 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc88e9-a2d6-4709-8f84-c01cad304fbf service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:58.283714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04872c1-1f48-4806-9735-459e60ea1a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ff87f1adad5487ab49f7f4ae8859df7, user_id=6dd9a208282d4a35a444069855b8b1d5, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:58.286626 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04872c1-1f48-4806-9735-459e60ea1a5a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:58.288561 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1284/2593] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:58 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:46:58.302318 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1310/2594] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:46:58 2022] PUT /identity/v3/projects/0ff87f1adad5487ab49f7f4ae8859df7/users/6dd9a208282d4a35a444069855b8b1d5/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:46:58.314437 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c8cc9-656a-419f-8a29-46ac283eaf75 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:58.314731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c8cc9-656a-419f-8a29-46ac283eaf75 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:58.315012 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c8cc9-656a-419f-8a29-46ac283eaf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:58.350691 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e50c8cc9-656a-419f-8a29-46ac283eaf75 None None] MFA Rules not processed for user `6dd9a208282d4a35a444069855b8b1d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:46:58.352148 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e50c8cc9-656a-419f-8a29-46ac283eaf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:58.425378 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1285/2595] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:46:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:46:58.437599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04872c1-1f48-4806-9735-459e60ea1a5a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:46:58.440151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c04872c1-1f48-4806-9735-459e60ea1a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:46:58.477159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:46:58.506322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04872c1-1f48-4806-9735-459e60ea1a5a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:46:58.510531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c931e-d802-4af0-b1d2-f4942fc75d06 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e053b6b25a4e4a20a05e6ffa6683222b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:46:58.511254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c931e-d802-4af0-b1d2-f4942fc75d06 service glance] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:46:58.511615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c931e-d802-4af0-b1d2-f4942fc75d06 service glance] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:46:58.512034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c931e-d802-4af0-b1d2-f4942fc75d06 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:46:58.521347 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c931e-d802-4af0-b1d2-f4942fc75d06 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:46:58.522820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c931e-d802-4af0-b1d2-f4942fc75d06 service glance] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:46:58.534338 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1311/2596] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:46:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:08.422139 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=67 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:08.528691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=13 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:11.511276 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c8cc9-656a-419f-8a29-46ac283eaf75 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:11.512742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e50c8cc9-656a-419f-8a29-46ac283eaf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:11.592793 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c8cc9-656a-419f-8a29-46ac283eaf75 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:11.598346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c7916-8490-448e-926b-934d51df3832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:11.599244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c7916-8490-448e-926b-934d51df3832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:11.599508 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c7916-8490-448e-926b-934d51df3832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:11.599785 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c7916-8490-448e-926b-934d51df3832 admin admin] PATH_INFO: `/v3/users/7d5083cc13fa4909bf5b4919aea5aede` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:11.601002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c7916-8490-448e-926b-934d51df3832 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d5083cc13fa4909bf5b4919aea5aede)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:11.602927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c7916-8490-448e-926b-934d51df3832 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:11.657609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e98c7916-8490-448e-926b-934d51df3832 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d5083cc13fa4909bf5b4919aea5aede'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:11.665368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e98c7916-8490-448e-926b-934d51df3832 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d5083cc13fa4909bf5b4919aea5aede'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:11.674157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e98c7916-8490-448e-926b-934d51df3832 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d5083cc13fa4909bf5b4919aea5aede'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:11.686167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e98c7916-8490-448e-926b-934d51df3832 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d5083cc13fa4909bf5b4919aea5aede'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:11.686892 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e98c7916-8490-448e-926b-934d51df3832 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d5083cc13fa4909bf5b4919aea5aede'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:11.692911 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1286/2597] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:47:11 2022] DELETE /identity/v3/users/7d5083cc13fa4909bf5b4919aea5aede => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:11.697767 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0c361-8732-4e43-b199-d4303e3af0f5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:11.698285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0c361-8732-4e43-b199-d4303e3af0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:11.698472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0c361-8732-4e43-b199-d4303e3af0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:11.723735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-89e0c361-8732-4e43-b199-d4303e3af0f5 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:11.725508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-89e0c361-8732-4e43-b199-d4303e3af0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:11.808047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1312/2598] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:11.818055 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c7916-8490-448e-926b-934d51df3832 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:11.820153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e98c7916-8490-448e-926b-934d51df3832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:11.881916 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c7916-8490-448e-926b-934d51df3832 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:11.885591 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce9de0-bddb-4a16-a468-9820acffccba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:11.886233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce9de0-bddb-4a16-a468-9820acffccba service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:11.886518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce9de0-bddb-4a16-a468-9820acffccba service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:11.886797 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce9de0-bddb-4a16-a468-9820acffccba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:11.896398 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce9de0-bddb-4a16-a468-9820acffccba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:11.897842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce9de0-bddb-4a16-a468-9820acffccba service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:11.907699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1287/2599] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:12.206762 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0c361-8732-4e43-b199-d4303e3af0f5 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:12.208274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-89e0c361-8732-4e43-b199-d4303e3af0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:12.221639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0c361-8732-4e43-b199-d4303e3af0f5 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:12.226370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:12.227235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:12.227517 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:12.227957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904 admin admin] PATH_INFO: `/v3/projects/223f82649e1e485da3fdbc8dabf96275` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:12.229145 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=223f82649e1e485da3fdbc8dabf96275)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:12.231081 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:12.280059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '223f82649e1e485da3fdbc8dabf96275', 'request_id': 'req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:12.286069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '223f82649e1e485da3fdbc8dabf96275', 'request_id': 'req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:12.292267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '223f82649e1e485da3fdbc8dabf96275', 'request_id': 'req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:12.305635 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904 admin admin] The token cache is being invalidate because project 223f82649e1e485da3fdbc8dabf96275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:47:12.305907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:12.306530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:12.308088 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1313/2600] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:47:12 2022] DELETE /identity/v3/projects/223f82649e1e485da3fdbc8dabf96275 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:12.312092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce9de0-bddb-4a16-a468-9820acffccba service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:12.322433 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce9de0-bddb-4a16-a468-9820acffccba service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:12.340330 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:12.380195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07172e-2d8b-4f7e-987e-770846e1120a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:12.380969 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07172e-2d8b-4f7e-987e-770846e1120a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:12.381251 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07172e-2d8b-4f7e-987e-770846e1120a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:12.381539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07172e-2d8b-4f7e-987e-770846e1120a admin admin] PATH_INFO: `/v3/users/17bdd737b4c942eeb42475a59f474cdb` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:12.382702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07172e-2d8b-4f7e-987e-770846e1120a admin admin] RBAC: Authorizing `identity:delete_user(user_id=17bdd737b4c942eeb42475a59f474cdb)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:12.384697 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07172e-2d8b-4f7e-987e-770846e1120a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:12.430360 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1a07172e-2d8b-4f7e-987e-770846e1120a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17bdd737b4c942eeb42475a59f474cdb'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:12.438283 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1a07172e-2d8b-4f7e-987e-770846e1120a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17bdd737b4c942eeb42475a59f474cdb'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:12.445355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1a07172e-2d8b-4f7e-987e-770846e1120a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17bdd737b4c942eeb42475a59f474cdb'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:12.458410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1a07172e-2d8b-4f7e-987e-770846e1120a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17bdd737b4c942eeb42475a59f474cdb'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:12.459196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1a07172e-2d8b-4f7e-987e-770846e1120a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17bdd737b4c942eeb42475a59f474cdb'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:12.465246 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1288/2601] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:47:12 2022] DELETE /identity/v3/users/17bdd737b4c942eeb42475a59f474cdb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:12.748746 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:12.750836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:12.773361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babb6a1-63dc-4cd6-9b7e-772a2e97a904 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:12.842994 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc582046-0889-45cc-b76a-077a3e42a14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:12.844830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc582046-0889-45cc-b76a-077a3e42a14f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:12.845171 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc582046-0889-45cc-b76a-077a3e42a14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:12.845450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc582046-0889-45cc-b76a-077a3e42a14f admin admin] PATH_INFO: `/v3/projects/4ea30662d0af420bb9688ea9b7e6192c` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:12.846649 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc582046-0889-45cc-b76a-077a3e42a14f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ea30662d0af420bb9688ea9b7e6192c)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:12.848618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc582046-0889-45cc-b76a-077a3e42a14f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:12.879649 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:12.898490 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bc582046-0889-45cc-b76a-077a3e42a14f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ea30662d0af420bb9688ea9b7e6192c', 'request_id': 'req-bc582046-0889-45cc-b76a-077a3e42a14f'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:12.903827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bc582046-0889-45cc-b76a-077a3e42a14f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ea30662d0af420bb9688ea9b7e6192c', 'request_id': 'req-bc582046-0889-45cc-b76a-077a3e42a14f'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:12.910884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bc582046-0889-45cc-b76a-077a3e42a14f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ea30662d0af420bb9688ea9b7e6192c', 'request_id': 'req-bc582046-0889-45cc-b76a-077a3e42a14f'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:12.923451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bc582046-0889-45cc-b76a-077a3e42a14f admin admin] The token cache is being invalidate because project 4ea30662d0af420bb9688ea9b7e6192c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:47:12.923746 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bc582046-0889-45cc-b76a-077a3e42a14f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:12.924414 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bc582046-0889-45cc-b76a-077a3e42a14f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:12.925957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1314/2602] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:47:12 2022] DELETE /identity/v3/projects/4ea30662d0af420bb9688ea9b7e6192c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:12.930853 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacc4af-083e-4f90-8066-d04b70dcda2d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:12.931335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacc4af-083e-4f90-8066-d04b70dcda2d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:12.931505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacc4af-083e-4f90-8066-d04b70dcda2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:12.950128 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-8eacc4af-083e-4f90-8066-d04b70dcda2d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:12.951338 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8eacc4af-083e-4f90-8066-d04b70dcda2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:13.012287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1289/2603] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:13.020323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9312fe-a421-4e20-9ff6-ecc46570b18d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:13.020620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9312fe-a421-4e20-9ff6-ecc46570b18d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:13.020928 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9312fe-a421-4e20-9ff6-ecc46570b18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:13.039342 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ab9312fe-a421-4e20-9ff6-ecc46570b18d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:13.040613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ab9312fe-a421-4e20-9ff6-ecc46570b18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:13.049800 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1315/2604] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:47:13.054059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacc4af-083e-4f90-8066-d04b70dcda2d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:13.064485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacc4af-083e-4f90-8066-d04b70dcda2d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:13.069028 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c096f-5d61-43f2-a2d5-ca6337cd481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:13.069679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c096f-5d61-43f2-a2d5-ca6337cd481c admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:13.069967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c096f-5d61-43f2-a2d5-ca6337cd481c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:13.070251 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c096f-5d61-43f2-a2d5-ca6337cd481c admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:13.070976 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c096f-5d61-43f2-a2d5-ca6337cd481c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:47:13.071287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c096f-5d61-43f2-a2d5-ca6337cd481c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:13.072957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c096f-5d61-43f2-a2d5-ca6337cd481c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:13.084532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1290/2605] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:47:13.088825 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9312fe-a421-4e20-9ff6-ecc46570b18d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:13.098069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9312fe-a421-4e20-9ff6-ecc46570b18d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:13.102624 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fe452-93d3-40d5-b2f0-8502fb72219b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:13.103263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fe452-93d3-40d5-b2f0-8502fb72219b admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:13.103545 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fe452-93d3-40d5-b2f0-8502fb72219b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:13.103816 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fe452-93d3-40d5-b2f0-8502fb72219b admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:13.104543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fe452-93d3-40d5-b2f0-8502fb72219b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:13.106431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fe452-93d3-40d5-b2f0-8502fb72219b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:13.132625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1316/2606] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:47:13 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:13.136755 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c096f-5d61-43f2-a2d5-ca6337cd481c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:13.147192 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c096f-5d61-43f2-a2d5-ca6337cd481c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:13.174885 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:13.207149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683d1a3-7174-4215-a169-93fe4c7aec2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:13.207808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683d1a3-7174-4215-a169-93fe4c7aec2c admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:13.208122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683d1a3-7174-4215-a169-93fe4c7aec2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:13.208397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683d1a3-7174-4215-a169-93fe4c7aec2c admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:13.209099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683d1a3-7174-4215-a169-93fe4c7aec2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:13.211026 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683d1a3-7174-4215-a169-93fe4c7aec2c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:13.231208 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1291/2607] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:47:13 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:47:13.235787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fe452-93d3-40d5-b2f0-8502fb72219b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:13.246798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fe452-93d3-40d5-b2f0-8502fb72219b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:13.252619 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf620e3-f850-41c3-94f3-e6c4f9a6e10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:13.253299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf620e3-f850-41c3-94f3-e6c4f9a6e10a admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:13.253596 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf620e3-f850-41c3-94f3-e6c4f9a6e10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:13.253869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf620e3-f850-41c3-94f3-e6c4f9a6e10a admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:13.254602 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf620e3-f850-41c3-94f3-e6c4f9a6e10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:13.256253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf620e3-f850-41c3-94f3-e6c4f9a6e10a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:13.267976 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1317/2608] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:47:13.272245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683d1a3-7174-4215-a169-93fe4c7aec2c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:13.283968 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683d1a3-7174-4215-a169-93fe4c7aec2c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:13.288809 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b3c6e-bed4-467b-b209-9e717f4e6375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:13.289479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b3c6e-bed4-467b-b209-9e717f4e6375 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:13.289765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b3c6e-bed4-467b-b209-9e717f4e6375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:13.290042 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b3c6e-bed4-467b-b209-9e717f4e6375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:13.290804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b3c6e-bed4-467b-b209-9e717f4e6375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:13.292650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b3c6e-bed4-467b-b209-9e717f4e6375 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:13.303856 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1292/2609] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:13.308196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf620e3-f850-41c3-94f3-e6c4f9a6e10a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:13.318995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf620e3-f850-41c3-94f3-e6c4f9a6e10a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:13.323677 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71e771-7558-4ac8-8abd-070613633a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:13.324594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71e771-7558-4ac8-8abd-070613633a3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:13.324939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71e771-7558-4ac8-8abd-070613633a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:13.325224 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71e771-7558-4ac8-8abd-070613633a3e admin admin] PATH_INFO: `/v3/projects/4e9f65c8618148f49c6b580c1af0b689/users/bb971377a9234445b173006c45bb8cce/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:13.342055 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71e771-7558-4ac8-8abd-070613633a3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e9f65c8618148f49c6b580c1af0b689, user_id=bb971377a9234445b173006c45bb8cce, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:13.345430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71e771-7558-4ac8-8abd-070613633a3e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:13.358974 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1318/2610] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:13 2022] PUT /identity/v3/projects/4e9f65c8618148f49c6b580c1af0b689/users/bb971377a9234445b173006c45bb8cce/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:13.370866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb820268-d930-422d-a071-159181e69873 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:13.371138 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb820268-d930-422d-a071-159181e69873 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:13.371420 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb820268-d930-422d-a071-159181e69873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:13.398882 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-cb820268-d930-422d-a071-159181e69873 None None] MFA Rules not processed for user `bb971377a9234445b173006c45bb8cce`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:13.400143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-cb820268-d930-422d-a071-159181e69873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:13.466984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1293/2611] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:13.472216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71e771-7558-4ac8-8abd-070613633a3e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:13.534842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71e771-7558-4ac8-8abd-070613633a3e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:13.539595 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82064a59-acfc-4f89-b572-c2370297b069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:13.540326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82064a59-acfc-4f89-b572-c2370297b069 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:13.540612 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82064a59-acfc-4f89-b572-c2370297b069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:13.540890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82064a59-acfc-4f89-b572-c2370297b069 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:13.541599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82064a59-acfc-4f89-b572-c2370297b069 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:13.543520 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82064a59-acfc-4f89-b572-c2370297b069 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:13.569494 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1319/2612] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:47:13 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:13.573408 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb820268-d930-422d-a071-159181e69873 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:13.584381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb820268-d930-422d-a071-159181e69873 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:13.641879 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943e86f-9a87-43ae-abac-c18c06fe71ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:13.642549 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943e86f-9a87-43ae-abac-c18c06fe71ed admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:13.642923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943e86f-9a87-43ae-abac-c18c06fe71ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:13.643102 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943e86f-9a87-43ae-abac-c18c06fe71ed admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:13.643799 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943e86f-9a87-43ae-abac-c18c06fe71ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:13.645766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943e86f-9a87-43ae-abac-c18c06fe71ed admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:13.668485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1294/2613] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:47:13 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:13.673377 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82064a59-acfc-4f89-b572-c2370297b069 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:13.683804 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82064a59-acfc-4f89-b572-c2370297b069 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:13.688747 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77351fa7-1f5c-4ec0-ac81-43faa8968ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:13.689524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77351fa7-1f5c-4ec0-ac81-43faa8968ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:13.689886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77351fa7-1f5c-4ec0-ac81-43faa8968ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:13.690284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77351fa7-1f5c-4ec0-ac81-43faa8968ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:13.691076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77351fa7-1f5c-4ec0-ac81-43faa8968ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:13.692860 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77351fa7-1f5c-4ec0-ac81-43faa8968ca0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:13.707508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1320/2614] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:13.708938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943e86f-9a87-43ae-abac-c18c06fe71ed admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:13.720644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943e86f-9a87-43ae-abac-c18c06fe71ed admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:13.725312 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7484f8-c66e-4c59-869b-9f506e2cc351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:13.726196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7484f8-c66e-4c59-869b-9f506e2cc351 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:13.726471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7484f8-c66e-4c59-869b-9f506e2cc351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:13.726737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7484f8-c66e-4c59-869b-9f506e2cc351 admin admin] PATH_INFO: `/v3/projects/8d526f189ca840b4b8aa6d35a4ccd663/users/dbb7da9c641346d0ae518f2431113e82/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:13.741800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7484f8-c66e-4c59-869b-9f506e2cc351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d526f189ca840b4b8aa6d35a4ccd663, user_id=dbb7da9c641346d0ae518f2431113e82, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:13.744563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7484f8-c66e-4c59-869b-9f506e2cc351 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:13.759677 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1295/2615] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:13 2022] PUT /identity/v3/projects/8d526f189ca840b4b8aa6d35a4ccd663/users/dbb7da9c641346d0ae518f2431113e82/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:13.763659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77351fa7-1f5c-4ec0-ac81-43faa8968ca0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:13.781082 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77351fa7-1f5c-4ec0-ac81-43faa8968ca0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:13.802450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:13.842050 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ec040-bcf6-49a5-aee3-b74f1157d352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:13.842717 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ec040-bcf6-49a5-aee3-b74f1157d352 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:13.843004 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ec040-bcf6-49a5-aee3-b74f1157d352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:13.843284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ec040-bcf6-49a5-aee3-b74f1157d352 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:13.844732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ec040-bcf6-49a5-aee3-b74f1157d352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:13.846386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ec040-bcf6-49a5-aee3-b74f1157d352 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:13.861039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1321/2616] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:13.864571 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7484f8-c66e-4c59-869b-9f506e2cc351 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:13.870312 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:13.877504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7484f8-c66e-4c59-869b-9f506e2cc351 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:13.882463 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4088caa-6d9b-4e15-b4df-44f723d5d2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:13.883363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4088caa-6d9b-4e15-b4df-44f723d5d2fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:13.883639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4088caa-6d9b-4e15-b4df-44f723d5d2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:13.883936 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4088caa-6d9b-4e15-b4df-44f723d5d2fb admin admin] PATH_INFO: `/v3/projects/8d526f189ca840b4b8aa6d35a4ccd663/users/dbb7da9c641346d0ae518f2431113e82/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:13.885878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4088caa-6d9b-4e15-b4df-44f723d5d2fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d526f189ca840b4b8aa6d35a4ccd663, user_id=dbb7da9c641346d0ae518f2431113e82, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:13.888588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4088caa-6d9b-4e15-b4df-44f723d5d2fb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:13.898927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1296/2617] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:13 2022] PUT /identity/v3/projects/8d526f189ca840b4b8aa6d35a4ccd663/users/dbb7da9c641346d0ae518f2431113e82/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:13.909089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c442b1-19a4-43fd-addf-f22640bf13c5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:13.909378 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c442b1-19a4-43fd-addf-f22640bf13c5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:13.909653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c442b1-19a4-43fd-addf-f22640bf13c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:13.936193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-74c442b1-19a4-43fd-addf-f22640bf13c5 None None] MFA Rules not processed for user `dbb7da9c641346d0ae518f2431113e82`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:13.938549 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-74c442b1-19a4-43fd-addf-f22640bf13c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:14.007820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1322/2618] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:14.018631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4088caa-6d9b-4e15-b4df-44f723d5d2fb admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:14.020410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e4088caa-6d9b-4e15-b4df-44f723d5d2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:14.087864 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4088caa-6d9b-4e15-b4df-44f723d5d2fb admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:14.091531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e18df-f052-426d-9b60-b6ccf97fa5a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:14.092208 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e18df-f052-426d-9b60-b6ccf97fa5a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:14.092495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e18df-f052-426d-9b60-b6ccf97fa5a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:14.092773 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e18df-f052-426d-9b60-b6ccf97fa5a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:14.149622 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e18df-f052-426d-9b60-b6ccf97fa5a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:14.151080 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e18df-f052-426d-9b60-b6ccf97fa5a8 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:14.160925 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1297/2619] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:24.004636 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=52 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:24.155895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:24.847025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c442b1-19a4-43fd-addf-f22640bf13c5 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:24.848440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-74c442b1-19a4-43fd-addf-f22640bf13c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:24.872666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c442b1-19a4-43fd-addf-f22640bf13c5 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:24.876948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85241ec9-192e-4a85-a138-de8cc0ff2aad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:24.877584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85241ec9-192e-4a85-a138-de8cc0ff2aad service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:24.877888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85241ec9-192e-4a85-a138-de8cc0ff2aad service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:24.878159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85241ec9-192e-4a85-a138-de8cc0ff2aad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:24.887246 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85241ec9-192e-4a85-a138-de8cc0ff2aad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:24.888790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85241ec9-192e-4a85-a138-de8cc0ff2aad service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:24.899399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1323/2620] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:25.229905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e18df-f052-426d-9b60-b6ccf97fa5a8 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:25.231523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-653e18df-f052-426d-9b60-b6ccf97fa5a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:25.303533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e18df-f052-426d-9b60-b6ccf97fa5a8 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:25.308097 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085aa479-7a00-445b-8135-bcc5aa9ce39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:25.308961 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085aa479-7a00-445b-8135-bcc5aa9ce39f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:25.309244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085aa479-7a00-445b-8135-bcc5aa9ce39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:25.309519 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085aa479-7a00-445b-8135-bcc5aa9ce39f admin admin] PATH_INFO: `/v3/users/e3ae8bc60a7449cea3da7e9f0e063f77` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:25.310699 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085aa479-7a00-445b-8135-bcc5aa9ce39f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3ae8bc60a7449cea3da7e9f0e063f77)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:25.313531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085aa479-7a00-445b-8135-bcc5aa9ce39f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:25.358283 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-085aa479-7a00-445b-8135-bcc5aa9ce39f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3ae8bc60a7449cea3da7e9f0e063f77'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:25.365910 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-085aa479-7a00-445b-8135-bcc5aa9ce39f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3ae8bc60a7449cea3da7e9f0e063f77'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:25.374083 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-085aa479-7a00-445b-8135-bcc5aa9ce39f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3ae8bc60a7449cea3da7e9f0e063f77'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:25.386618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-085aa479-7a00-445b-8135-bcc5aa9ce39f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3ae8bc60a7449cea3da7e9f0e063f77'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:25.387316 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-085aa479-7a00-445b-8135-bcc5aa9ce39f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3ae8bc60a7449cea3da7e9f0e063f77'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:25.394324 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1298/2621] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:47:25 2022] DELETE /identity/v3/users/e3ae8bc60a7449cea3da7e9f0e063f77 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:25.398893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa93f0-16ea-4b2a-8b8f-8c133100bca1 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:25.399420 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa93f0-16ea-4b2a-8b8f-8c133100bca1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:25.399420 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa93f0-16ea-4b2a-8b8f-8c133100bca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:25.417224 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-57fa93f0-16ea-4b2a-8b8f-8c133100bca1 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:25.418421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-57fa93f0-16ea-4b2a-8b8f-8c133100bca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:25.477368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1324/2622] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:25.487217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085aa479-7a00-445b-8135-bcc5aa9ce39f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:25.488683 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-085aa479-7a00-445b-8135-bcc5aa9ce39f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:25.546160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085aa479-7a00-445b-8135-bcc5aa9ce39f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:25.550092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70804e5-ddea-4891-914a-d3b29f67ccae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:25.550734 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70804e5-ddea-4891-914a-d3b29f67ccae service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:25.551019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70804e5-ddea-4891-914a-d3b29f67ccae service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:25.551296 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70804e5-ddea-4891-914a-d3b29f67ccae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:25.560826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70804e5-ddea-4891-914a-d3b29f67ccae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:25.562920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70804e5-ddea-4891-914a-d3b29f67ccae service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:25.572607 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1299/2623] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:25.851497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa93f0-16ea-4b2a-8b8f-8c133100bca1 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:25.852995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-57fa93f0-16ea-4b2a-8b8f-8c133100bca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:25.866024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa93f0-16ea-4b2a-8b8f-8c133100bca1 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:25.871059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ea623-c1f6-424a-a3e7-44e9fe85d697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:25.871955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ea623-c1f6-424a-a3e7-44e9fe85d697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:25.872246 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ea623-c1f6-424a-a3e7-44e9fe85d697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:25.872528 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ea623-c1f6-424a-a3e7-44e9fe85d697 admin admin] PATH_INFO: `/v3/projects/6f07628e8d854e99a072e66429d1b950` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:25.873771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ea623-c1f6-424a-a3e7-44e9fe85d697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f07628e8d854e99a072e66429d1b950)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:25.876322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ea623-c1f6-424a-a3e7-44e9fe85d697 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:25.935029 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-df9ea623-c1f6-424a-a3e7-44e9fe85d697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f07628e8d854e99a072e66429d1b950', 'request_id': 'req-df9ea623-c1f6-424a-a3e7-44e9fe85d697'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:25.941267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-df9ea623-c1f6-424a-a3e7-44e9fe85d697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f07628e8d854e99a072e66429d1b950', 'request_id': 'req-df9ea623-c1f6-424a-a3e7-44e9fe85d697'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:25.947565 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-df9ea623-c1f6-424a-a3e7-44e9fe85d697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f07628e8d854e99a072e66429d1b950', 'request_id': 'req-df9ea623-c1f6-424a-a3e7-44e9fe85d697'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:25.960535 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-df9ea623-c1f6-424a-a3e7-44e9fe85d697 admin admin] The token cache is being invalidate because project 6f07628e8d854e99a072e66429d1b950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:47:25.960886 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-df9ea623-c1f6-424a-a3e7-44e9fe85d697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:25.961588 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-df9ea623-c1f6-424a-a3e7-44e9fe85d697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:25.963319 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1325/2624] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:47:25 2022] DELETE /identity/v3/projects/6f07628e8d854e99a072e66429d1b950 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:25.968486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac43398-febb-4cb8-bee3-a775dd57e70f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:25.968777 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac43398-febb-4cb8-bee3-a775dd57e70f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:25.969054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac43398-febb-4cb8-bee3-a775dd57e70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:25.987188 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-dac43398-febb-4cb8-bee3-a775dd57e70f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:25.988428 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dac43398-febb-4cb8-bee3-a775dd57e70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:26.005985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:26.050499 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1300/2625] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:47:26.058591 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b7cb4-f971-4349-a365-33a0f532b2de None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:26.058885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b7cb4-f971-4349-a365-33a0f532b2de None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:26.059167 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b7cb4-f971-4349-a365-33a0f532b2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:26.078229 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b03b7cb4-f971-4349-a365-33a0f532b2de None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:26.079459 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b03b7cb4-f971-4349-a365-33a0f532b2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:26.087396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1326/2626] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:26.091573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac43398-febb-4cb8-bee3-a775dd57e70f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:26.102125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac43398-febb-4cb8-bee3-a775dd57e70f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:26.106572 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d90b1-e74c-42f1-a5cd-1a4f36476c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:26.107214 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d90b1-e74c-42f1-a5cd-1a4f36476c88 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:26.107491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d90b1-e74c-42f1-a5cd-1a4f36476c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:26.107762 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d90b1-e74c-42f1-a5cd-1a4f36476c88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:26.108521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d90b1-e74c-42f1-a5cd-1a4f36476c88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:47:26.109484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d90b1-e74c-42f1-a5cd-1a4f36476c88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:26.111063 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d90b1-e74c-42f1-a5cd-1a4f36476c88 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:26.122869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1301/2627] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:47:26.127379 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b7cb4-f971-4349-a365-33a0f532b2de None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:26.136732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b7cb4-f971-4349-a365-33a0f532b2de None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:26.141799 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d677e1c-7086-49fb-81df-64e7313489b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:26.142445 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d677e1c-7086-49fb-81df-64e7313489b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:26.142734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d677e1c-7086-49fb-81df-64e7313489b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:26.143010 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d677e1c-7086-49fb-81df-64e7313489b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:26.143704 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d677e1c-7086-49fb-81df-64e7313489b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:26.146311 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d677e1c-7086-49fb-81df-64e7313489b4 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:26.177900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1327/2628] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:47:26 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:26.182203 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d90b1-e74c-42f1-a5cd-1a4f36476c88 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:26.193485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d90b1-e74c-42f1-a5cd-1a4f36476c88 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:26.265628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fc946-02a2-47cb-a575-fd46db7f8bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:26.266347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fc946-02a2-47cb-a575-fd46db7f8bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:26.266480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fc946-02a2-47cb-a575-fd46db7f8bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:26.266850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fc946-02a2-47cb-a575-fd46db7f8bf1 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:26.267588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fc946-02a2-47cb-a575-fd46db7f8bf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:26.269628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fc946-02a2-47cb-a575-fd46db7f8bf1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:26.291854 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1302/2629] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:47:26 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:26.294948 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d677e1c-7086-49fb-81df-64e7313489b4 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:26.307504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d677e1c-7086-49fb-81df-64e7313489b4 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:26.312998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6b237-85e0-4fa7-bcae-8237a41cd9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:26.313693 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6b237-85e0-4fa7-bcae-8237a41cd9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:26.313977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6b237-85e0-4fa7-bcae-8237a41cd9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:26.314864 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6b237-85e0-4fa7-bcae-8237a41cd9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:26.315742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6b237-85e0-4fa7-bcae-8237a41cd9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:26.317240 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6b237-85e0-4fa7-bcae-8237a41cd9ee admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:26.331968 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1328/2630] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:26.333537 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fc946-02a2-47cb-a575-fd46db7f8bf1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:26.346400 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fc946-02a2-47cb-a575-fd46db7f8bf1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:26.351145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b479e7-9a12-4f47-b564-0de0d2b6d1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:26.351814 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b479e7-9a12-4f47-b564-0de0d2b6d1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:26.352194 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b479e7-9a12-4f47-b564-0de0d2b6d1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:26.352472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b479e7-9a12-4f47-b564-0de0d2b6d1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:26.353161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b479e7-9a12-4f47-b564-0de0d2b6d1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:26.354789 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b479e7-9a12-4f47-b564-0de0d2b6d1d1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:26.369875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1303/2631] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:26.371236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6b237-85e0-4fa7-bcae-8237a41cd9ee admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:26.382543 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6b237-85e0-4fa7-bcae-8237a41cd9ee admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:26.387287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd68145-566d-4b5e-a48d-8444571454b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:26.388343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd68145-566d-4b5e-a48d-8444571454b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:26.388628 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd68145-566d-4b5e-a48d-8444571454b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:26.388904 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd68145-566d-4b5e-a48d-8444571454b3 admin admin] PATH_INFO: `/v3/projects/cff5412db5c24a03a4ca18ec2780d77c/users/9d439ed3cd1f4610a4d4902b42d19f37/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:26.405615 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd68145-566d-4b5e-a48d-8444571454b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff5412db5c24a03a4ca18ec2780d77c, user_id=9d439ed3cd1f4610a4d4902b42d19f37, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:26.409513 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd68145-566d-4b5e-a48d-8444571454b3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:26.413075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:26.428571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1329/2632] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:26 2022] PUT /identity/v3/projects/cff5412db5c24a03a4ca18ec2780d77c/users/9d439ed3cd1f4610a4d4902b42d19f37/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:47:26.434101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f347e-542c-490f-9168-6176a58d2672 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:26.434317 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f347e-542c-490f-9168-6176a58d2672 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:26.434626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f347e-542c-490f-9168-6176a58d2672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:26.454291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-aa9f347e-542c-490f-9168-6176a58d2672 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:26.455510 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-aa9f347e-542c-490f-9168-6176a58d2672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:26.519163 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1304/2633] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:26.528872 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd68145-566d-4b5e-a48d-8444571454b3 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:26.534534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5cd68145-566d-4b5e-a48d-8444571454b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:26.601958 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd68145-566d-4b5e-a48d-8444571454b3 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:26.605804 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30fccb-4dc6-4abc-a70a-f347d5bc86cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:26.606393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30fccb-4dc6-4abc-a70a-f347d5bc86cc service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:26.606682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30fccb-4dc6-4abc-a70a-f347d5bc86cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:26.606965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30fccb-4dc6-4abc-a70a-f347d5bc86cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:26.617254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30fccb-4dc6-4abc-a70a-f347d5bc86cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:26.618703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30fccb-4dc6-4abc-a70a-f347d5bc86cc service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:26.630144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1330/2634] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:30.344272 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a4042-2b1e-4b77-b0e0-52618052405d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:30.345198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a4042-2b1e-4b77-b0e0-52618052405d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:30.345661 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a4042-2b1e-4b77-b0e0-52618052405d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:30.386488 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:30.411881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-387a4042-2b1e-4b77-b0e0-52618052405d None None] MFA Rules not processed for user `257450db238d4efda783e30cbcd23d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:30.413747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-387a4042-2b1e-4b77-b0e0-52618052405d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:30.511452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1305/2635] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jul 21 09:47:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:30.587417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30fccb-4dc6-4abc-a70a-f347d5bc86cc service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:30.589967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6b30fccb-4dc6-4abc-a70a-f347d5bc86cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:30.722021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30fccb-4dc6-4abc-a70a-f347d5bc86cc service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:30.740149 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57542c51-1839-405a-83ee-d3d276365108 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061def74ec7f42ce9a174e0944bf499f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:30.740935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57542c51-1839-405a-83ee-d3d276365108 service swift] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:30.741228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57542c51-1839-405a-83ee-d3d276365108 service swift] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:30.741513 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57542c51-1839-405a-83ee-d3d276365108 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:30.756560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57542c51-1839-405a-83ee-d3d276365108 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:30.760872 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57542c51-1839-405a-83ee-d3d276365108 service swift] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:30.779867 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1331/2636] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jul 21 09:47:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 09:47:30.992167 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a4042-2b1e-4b77-b0e0-52618052405d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:30.994216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-387a4042-2b1e-4b77-b0e0-52618052405d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:31.023500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a4042-2b1e-4b77-b0e0-52618052405d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:31.029443 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b41438-fed6-46e7-af5e-a87ea73afebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:31.030900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b41438-fed6-46e7-af5e-a87ea73afebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:31.031380 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b41438-fed6-46e7-af5e-a87ea73afebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:31.031827 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b41438-fed6-46e7-af5e-a87ea73afebb admin admin] PATH_INFO: `/v3/users/bb971377a9234445b173006c45bb8cce` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:31.033630 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b41438-fed6-46e7-af5e-a87ea73afebb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb971377a9234445b173006c45bb8cce)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:31.037518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b41438-fed6-46e7-af5e-a87ea73afebb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:31.126735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-03b41438-fed6-46e7-af5e-a87ea73afebb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb971377a9234445b173006c45bb8cce'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:31.148755 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-03b41438-fed6-46e7-af5e-a87ea73afebb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb971377a9234445b173006c45bb8cce'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:31.168747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-03b41438-fed6-46e7-af5e-a87ea73afebb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb971377a9234445b173006c45bb8cce'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:31.190449 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-03b41438-fed6-46e7-af5e-a87ea73afebb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb971377a9234445b173006c45bb8cce'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:31.196830 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-03b41438-fed6-46e7-af5e-a87ea73afebb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb971377a9234445b173006c45bb8cce'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:31.208020 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1306/2637] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:47:30 2022] DELETE /identity/v3/users/bb971377a9234445b173006c45bb8cce => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:31.209946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e011e0c-00e8-4926-a912-e811e402e871 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:31.210492 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e011e0c-00e8-4926-a912-e811e402e871 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:31.210776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e011e0c-00e8-4926-a912-e811e402e871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:31.239460 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0e011e0c-00e8-4926-a912-e811e402e871 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:31.240818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0e011e0c-00e8-4926-a912-e811e402e871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:31.366952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b41438-fed6-46e7-af5e-a87ea73afebb admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:31.368493 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-03b41438-fed6-46e7-af5e-a87ea73afebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:31.378345 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1332/2638] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:31.457957 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b41438-fed6-46e7-af5e-a87ea73afebb admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:31.469825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b656b3-59e0-4514-b11d-7d4c18c21d03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:31.470537 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b656b3-59e0-4514-b11d-7d4c18c21d03 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:31.471166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b656b3-59e0-4514-b11d-7d4c18c21d03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:31.471166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b656b3-59e0-4514-b11d-7d4c18c21d03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:31.494556 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b656b3-59e0-4514-b11d-7d4c18c21d03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:31.497054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b656b3-59e0-4514-b11d-7d4c18c21d03 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:31.512143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1307/2639] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:47:31.641013 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac17b4-3f82-49ef-ab4a-6f4218f790f6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:31.641788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac17b4-3f82-49ef-ab4a-6f4218f790f6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:31.641984 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac17b4-3f82-49ef-ab4a-6f4218f790f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:31.677797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d2ac17b4-3f82-49ef-ab4a-6f4218f790f6 None None] MFA Rules not processed for user `257450db238d4efda783e30cbcd23d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:31.693121 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d2ac17b4-3f82-49ef-ab4a-6f4218f790f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:31.701554 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:31.756962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19953c-d441-47f8-89ad-f91a4dd437ef None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:31.757424 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19953c-d441-47f8-89ad-f91a4dd437ef None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:31.757719 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19953c-d441-47f8-89ad-f91a4dd437ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:31.778044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:31.811466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1333/2640] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jul 21 09:47:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:31.815188 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5b19953c-d441-47f8-89ad-f91a4dd437ef None None] MFA Rules not processed for user `9d439ed3cd1f4610a4d4902b42d19f37`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:31.816489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5b19953c-d441-47f8-89ad-f91a4dd437ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:31.827978 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac17b4-3f82-49ef-ab4a-6f4218f790f6 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:31.829947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d2ac17b4-3f82-49ef-ab4a-6f4218f790f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:31.914678 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1308/2641] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:31.919554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19953c-d441-47f8-89ad-f91a4dd437ef None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:31.921905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5b19953c-d441-47f8-89ad-f91a4dd437ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:31.949919 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19953c-d441-47f8-89ad-f91a4dd437ef None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:31.956979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9566e-a905-41bd-9040-77aefd73ea71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:31.958599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9566e-a905-41bd-9040-77aefd73ea71 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:31.960060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9566e-a905-41bd-9040-77aefd73ea71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:31.960597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9566e-a905-41bd-9040-77aefd73ea71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:31.962125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a9566e-a905-41bd-9040-77aefd73ea71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:31.964659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac17b4-3f82-49ef-ab4a-6f4218f790f6 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:31.965303 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a9566e-a905-41bd-9040-77aefd73ea71 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:31.971666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac329f6-59d5-46f9-9794-c65856a29fdd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061def74ec7f42ce9a174e0944bf499f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:31.972490 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac329f6-59d5-46f9-9794-c65856a29fdd service swift] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:31.972776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac329f6-59d5-46f9-9794-c65856a29fdd service swift] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:31.973045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac329f6-59d5-46f9-9794-c65856a29fdd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:32.001696 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac329f6-59d5-46f9-9794-c65856a29fdd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:32.003327 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac329f6-59d5-46f9-9794-c65856a29fdd service swift] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:32.004889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1309/2642] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:47:31 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:32.005837 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9566e-a905-41bd-9040-77aefd73ea71 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:32.007817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d2a9566e-a905-41bd-9040-77aefd73ea71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:32.032341 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1334/2643] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jul 21 09:47:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 09:47:32.033362 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac329f6-59d5-46f9-9794-c65856a29fdd service swift] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:32.054020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac329f6-59d5-46f9-9794-c65856a29fdd service swift] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:32.135783 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9566e-a905-41bd-9040-77aefd73ea71 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:32.143664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd8fe0-f03e-457d-81de-107e113756be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:32.144903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd8fe0-f03e-457d-81de-107e113756be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:32.145188 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd8fe0-f03e-457d-81de-107e113756be admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:32.145466 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd8fe0-f03e-457d-81de-107e113756be admin admin] PATH_INFO: `/v3/projects/4e9f65c8618148f49c6b580c1af0b689` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:32.147006 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd8fe0-f03e-457d-81de-107e113756be admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e9f65c8618148f49c6b580c1af0b689)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:32.151863 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd8fe0-f03e-457d-81de-107e113756be admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:32.168603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e1dd1-ef61-466f-97fa-d17036ec9e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:32.170210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e1dd1-ef61-466f-97fa-d17036ec9e87 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:32.170210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e1dd1-ef61-466f-97fa-d17036ec9e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:32.170522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e1dd1-ef61-466f-97fa-d17036ec9e87 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:32.171812 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e1dd1-ef61-466f-97fa-d17036ec9e87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:32.175894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e1dd1-ef61-466f-97fa-d17036ec9e87 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:32.217521 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1335/2644] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:47:32 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:32.222832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e1dd1-ef61-466f-97fa-d17036ec9e87 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:32.241041 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e1dd1-ef61-466f-97fa-d17036ec9e87 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:32.260513 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-01fd8fe0-f03e-457d-81de-107e113756be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e9f65c8618148f49c6b580c1af0b689', 'request_id': 'req-01fd8fe0-f03e-457d-81de-107e113756be'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:32.281214 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-01fd8fe0-f03e-457d-81de-107e113756be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e9f65c8618148f49c6b580c1af0b689', 'request_id': 'req-01fd8fe0-f03e-457d-81de-107e113756be'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:32.293522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-01fd8fe0-f03e-457d-81de-107e113756be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e9f65c8618148f49c6b580c1af0b689', 'request_id': 'req-01fd8fe0-f03e-457d-81de-107e113756be'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:32.311466 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-01fd8fe0-f03e-457d-81de-107e113756be admin admin] The token cache is being invalidate because project 4e9f65c8618148f49c6b580c1af0b689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:47:32.312368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-01fd8fe0-f03e-457d-81de-107e113756be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:32.313066 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-01fd8fe0-f03e-457d-81de-107e113756be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:32.316998 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1310/2645] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:47:32 2022] DELETE /identity/v3/projects/4e9f65c8618148f49c6b580c1af0b689 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:32.319893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:32.320840 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd8fe0-f03e-457d-81de-107e113756be admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:32.323158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-01fd8fe0-f03e-457d-81de-107e113756be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:32.338910 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:32.370959 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a3ec9-e8e1-4a69-a068-3dc3ff2fc691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:32.371742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a3ec9-e8e1-4a69-a068-3dc3ff2fc691 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:32.372220 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a3ec9-e8e1-4a69-a068-3dc3ff2fc691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:32.372739 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a3ec9-e8e1-4a69-a068-3dc3ff2fc691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:32.373380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a3ec9-e8e1-4a69-a068-3dc3ff2fc691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:32.376155 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a3ec9-e8e1-4a69-a068-3dc3ff2fc691 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:32.392485 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1336/2646] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:32.394169 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6d421-26f0-44a1-aaf1-7aeef17e9e25 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:32.394329 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6d421-26f0-44a1-aaf1-7aeef17e9e25 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:32.394735 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6d421-26f0-44a1-aaf1-7aeef17e9e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:32.413279 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-54c6d421-26f0-44a1-aaf1-7aeef17e9e25 None None] MFA Rules not processed for user `257450db238d4efda783e30cbcd23d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:32.414593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-54c6d421-26f0-44a1-aaf1-7aeef17e9e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:32.422525 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd8fe0-f03e-457d-81de-107e113756be admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:32.427359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd8349-d5b7-4289-933d-389b0649790f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:32.428290 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd8349-d5b7-4289-933d-389b0649790f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:32.428573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd8349-d5b7-4289-933d-389b0649790f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:32.428845 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd8349-d5b7-4289-933d-389b0649790f admin admin] PATH_INFO: `/v3/users/dbb7da9c641346d0ae518f2431113e82` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:32.430068 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd8349-d5b7-4289-933d-389b0649790f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbb7da9c641346d0ae518f2431113e82)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:32.432059 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd8349-d5b7-4289-933d-389b0649790f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:32.479241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1337/2647] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jul 21 09:47:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:32.480022 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6d421-26f0-44a1-aaf1-7aeef17e9e25 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:32.481400 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-54c6d421-26f0-44a1-aaf1-7aeef17e9e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:32.482966 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a8cd8349-d5b7-4289-933d-389b0649790f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbb7da9c641346d0ae518f2431113e82'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:32.492795 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a8cd8349-d5b7-4289-933d-389b0649790f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbb7da9c641346d0ae518f2431113e82'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:32.498077 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6d421-26f0-44a1-aaf1-7aeef17e9e25 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:32.502142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a8cd8349-d5b7-4289-933d-389b0649790f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbb7da9c641346d0ae518f2431113e82'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:32.503466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff3966-d59c-44a1-abfd-8ff7e5d6b7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:32.504483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff3966-d59c-44a1-abfd-8ff7e5d6b7bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:32.504843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff3966-d59c-44a1-abfd-8ff7e5d6b7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:32.505193 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff3966-d59c-44a1-abfd-8ff7e5d6b7bb admin admin] PATH_INFO: `/v3/projects/348dca2fbc924780808fa77de996a3fe/users/86830104a3524a30aea4d5c5ecc241dc/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:32.517396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a8cd8349-d5b7-4289-933d-389b0649790f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbb7da9c641346d0ae518f2431113e82'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:32.518459 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a8cd8349-d5b7-4289-933d-389b0649790f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbb7da9c641346d0ae518f2431113e82'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:32.521019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff3966-d59c-44a1-abfd-8ff7e5d6b7bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=348dca2fbc924780808fa77de996a3fe, user_id=86830104a3524a30aea4d5c5ecc241dc, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:32.523706 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff3966-d59c-44a1-abfd-8ff7e5d6b7bb admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:32.524737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1311/2648] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:47:32 2022] DELETE /identity/v3/users/dbb7da9c641346d0ae518f2431113e82 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:32.525522 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd8349-d5b7-4289-933d-389b0649790f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:32.526936 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a8cd8349-d5b7-4289-933d-389b0649790f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:32.542821 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1338/2649] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:32 2022] PUT /identity/v3/projects/348dca2fbc924780808fa77de996a3fe/users/86830104a3524a30aea4d5c5ecc241dc/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:32.547409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff3966-d59c-44a1-abfd-8ff7e5d6b7bb admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:32.549243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-31ff3966-d59c-44a1-abfd-8ff7e5d6b7bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:32.613322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd8349-d5b7-4289-933d-389b0649790f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:32.636953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff3966-d59c-44a1-abfd-8ff7e5d6b7bb admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:32.642551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec39e1f-53df-4b4a-9e0f-7d8a79af9b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:32.643412 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec39e1f-53df-4b4a-9e0f-7d8a79af9b0d admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:32.644132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec39e1f-53df-4b4a-9e0f-7d8a79af9b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:32.644551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:32.644681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec39e1f-53df-4b4a-9e0f-7d8a79af9b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:32.645750 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec39e1f-53df-4b4a-9e0f-7d8a79af9b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:32.647762 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec39e1f-53df-4b4a-9e0f-7d8a79af9b0d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:32.661086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1339/2650] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:47:32.665304 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec39e1f-53df-4b4a-9e0f-7d8a79af9b0d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:32.676958 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec39e1f-53df-4b4a-9e0f-7d8a79af9b0d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:32.681935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ce366-804b-4060-9152-b578aac13b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:32.682963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ce366-804b-4060-9152-b578aac13b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:32.683320 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ce366-804b-4060-9152-b578aac13b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:32.683681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ce366-804b-4060-9152-b578aac13b26 admin admin] PATH_INFO: `/v3/projects/348dca2fbc924780808fa77de996a3fe/users/86830104a3524a30aea4d5c5ecc241dc/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:32.685773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ce366-804b-4060-9152-b578aac13b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=348dca2fbc924780808fa77de996a3fe, user_id=86830104a3524a30aea4d5c5ecc241dc, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:32.689128 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ce366-804b-4060-9152-b578aac13b26 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:32.693510 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f839e34-2ed1-4bb0-ad84-b3f6a7fe8f7d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061def74ec7f42ce9a174e0944bf499f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:32.693510 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f839e34-2ed1-4bb0-ad84-b3f6a7fe8f7d service swift] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:32.701069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1340/2651] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:32 2022] PUT /identity/v3/projects/348dca2fbc924780808fa77de996a3fe/users/86830104a3524a30aea4d5c5ecc241dc/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:32.704460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f839e34-2ed1-4bb0-ad84-b3f6a7fe8f7d service swift] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:32.704884 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f839e34-2ed1-4bb0-ad84-b3f6a7fe8f7d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:32.713462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8f839e34-2ed1-4bb0-ad84-b3f6a7fe8f7d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:32.808657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f839e34-2ed1-4bb0-ad84-b3f6a7fe8f7d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:32.810403 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f839e34-2ed1-4bb0-ad84-b3f6a7fe8f7d service swift] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:32.821459 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1312/2652] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jul 21 09:47:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 09:47:32.865302 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ce366-804b-4060-9152-b578aac13b26 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:32.867782 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-643ce366-804b-4060-9152-b578aac13b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:32.934370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:32.994000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ce366-804b-4060-9152-b578aac13b26 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:33.000486 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80784571-e83a-4989-8d3e-77ed0ac0caa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:33.001587 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80784571-e83a-4989-8d3e-77ed0ac0caa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:33.002035 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80784571-e83a-4989-8d3e-77ed0ac0caa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:33.002482 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80784571-e83a-4989-8d3e-77ed0ac0caa6 admin admin] PATH_INFO: `/v3/projects/8d526f189ca840b4b8aa6d35a4ccd663` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:33.004459 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80784571-e83a-4989-8d3e-77ed0ac0caa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d526f189ca840b4b8aa6d35a4ccd663)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:33.006390 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80784571-e83a-4989-8d3e-77ed0ac0caa6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:33.079268 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-80784571-e83a-4989-8d3e-77ed0ac0caa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d526f189ca840b4b8aa6d35a4ccd663', 'request_id': 'req-80784571-e83a-4989-8d3e-77ed0ac0caa6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:33.092342 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-80784571-e83a-4989-8d3e-77ed0ac0caa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d526f189ca840b4b8aa6d35a4ccd663', 'request_id': 'req-80784571-e83a-4989-8d3e-77ed0ac0caa6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:33.107307 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-80784571-e83a-4989-8d3e-77ed0ac0caa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d526f189ca840b4b8aa6d35a4ccd663', 'request_id': 'req-80784571-e83a-4989-8d3e-77ed0ac0caa6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:33.128395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-80784571-e83a-4989-8d3e-77ed0ac0caa6 admin admin] The token cache is being invalidate because project 8d526f189ca840b4b8aa6d35a4ccd663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:47:33.128712 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-80784571-e83a-4989-8d3e-77ed0ac0caa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:33.129977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-80784571-e83a-4989-8d3e-77ed0ac0caa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:33.131568 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1341/2653] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:47:32 2022] DELETE /identity/v3/projects/8d526f189ca840b4b8aa6d35a4ccd663 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:33.137519 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3ef61-c378-4fbe-80cb-3cf07d47e3ae None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:33.138098 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3ef61-c378-4fbe-80cb-3cf07d47e3ae None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:33.138541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3ef61-c378-4fbe-80cb-3cf07d47e3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:33.158495 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-45b3ef61-c378-4fbe-80cb-3cf07d47e3ae None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:33.159592 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-45b3ef61-c378-4fbe-80cb-3cf07d47e3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:33.261692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1313/2654] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:47:33.270762 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd2c26-5a4b-4bcf-8c12-a4fc63115045 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:33.271055 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd2c26-5a4b-4bcf-8c12-a4fc63115045 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:33.271329 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd2c26-5a4b-4bcf-8c12-a4fc63115045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:33.272239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a46b76-fda8-474f-bb41-ad3fe1e284cc None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:33.272797 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a46b76-fda8-474f-bb41-ad3fe1e284cc None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:33.273746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a46b76-fda8-474f-bb41-ad3fe1e284cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:33.292464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-34fd2c26-5a4b-4bcf-8c12-a4fc63115045 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:33.293639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-34fd2c26-5a4b-4bcf-8c12-a4fc63115045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:33.305651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-35a46b76-fda8-474f-bb41-ad3fe1e284cc None None] MFA Rules not processed for user `257450db238d4efda783e30cbcd23d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:33.307018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-35a46b76-fda8-474f-bb41-ad3fe1e284cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:33.308138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1342/2655] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:33.312686 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd2c26-5a4b-4bcf-8c12-a4fc63115045 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:33.323310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd2c26-5a4b-4bcf-8c12-a4fc63115045 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:33.329716 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c48fe-12d7-4a43-8762-355224d6d59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:33.330448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c48fe-12d7-4a43-8762-355224d6d59d admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:33.330702 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c48fe-12d7-4a43-8762-355224d6d59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:33.331216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c48fe-12d7-4a43-8762-355224d6d59d admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:33.331791 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c48fe-12d7-4a43-8762-355224d6d59d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:47:33.332142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c48fe-12d7-4a43-8762-355224d6d59d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:33.333789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c48fe-12d7-4a43-8762-355224d6d59d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:33.347880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1343/2656] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 09:47:33.352366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c48fe-12d7-4a43-8762-355224d6d59d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:33.363674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c48fe-12d7-4a43-8762-355224d6d59d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:33.368594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035af1ca-986e-4bb4-9c02-4373805be6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:33.369324 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035af1ca-986e-4bb4-9c02-4373805be6fd admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:33.369619 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035af1ca-986e-4bb4-9c02-4373805be6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:33.369902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035af1ca-986e-4bb4-9c02-4373805be6fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:33.370791 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035af1ca-986e-4bb4-9c02-4373805be6fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:33.372657 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035af1ca-986e-4bb4-9c02-4373805be6fd admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:33.391003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1314/2657] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jul 21 09:47:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:47:33.404876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1344/2658] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:47:33 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:33.408513 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a46b76-fda8-474f-bb41-ad3fe1e284cc None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:33.410113 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035af1ca-986e-4bb4-9c02-4373805be6fd admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:33.411438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-35a46b76-fda8-474f-bb41-ad3fe1e284cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:33.428152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:33.499456 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a46b76-fda8-474f-bb41-ad3fe1e284cc None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:33.516395 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035af1ca-986e-4bb4-9c02-4373805be6fd admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:33.523447 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d087c6a-c7b0-4690-b07e-02487a399d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:33.524187 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d087c6a-c7b0-4690-b07e-02487a399d25 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:33.524485 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d087c6a-c7b0-4690-b07e-02487a399d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:33.524768 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d087c6a-c7b0-4690-b07e-02487a399d25 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:33.526693 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d087c6a-c7b0-4690-b07e-02487a399d25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:33.529626 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d087c6a-c7b0-4690-b07e-02487a399d25 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:33.567967 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1345/2659] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:47:33 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:33.570297 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d087c6a-c7b0-4690-b07e-02487a399d25 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:33.591368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d087c6a-c7b0-4690-b07e-02487a399d25 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:33.598866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e3eaa-5d9d-4f69-848a-44e4fc8d163d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:33.599935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e3eaa-5d9d-4f69-848a-44e4fc8d163d admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:33.600346 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e3eaa-5d9d-4f69-848a-44e4fc8d163d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:33.600729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e3eaa-5d9d-4f69-848a-44e4fc8d163d admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:33.600875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1f1fb-b4d8-415e-be56-ce39e2a04541 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061def74ec7f42ce9a174e0944bf499f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:33.601688 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e3eaa-5d9d-4f69-848a-44e4fc8d163d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:33.602053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1f1fb-b4d8-415e-be56-ce39e2a04541 service swift] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:33.602531 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1f1fb-b4d8-415e-be56-ce39e2a04541 service swift] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:33.603425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1f1fb-b4d8-415e-be56-ce39e2a04541 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:33.603734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e3eaa-5d9d-4f69-848a-44e4fc8d163d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:33.614257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:33.626457 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1346/2660] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:47:33.627643 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e3eaa-5d9d-4f69-848a-44e4fc8d163d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:33.645232 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e3eaa-5d9d-4f69-848a-44e4fc8d163d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:33.652158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61eb76-0bfa-4789-8f87-012d4adff24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:33.653396 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61eb76-0bfa-4789-8f87-012d4adff24d admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:33.654273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61eb76-0bfa-4789-8f87-012d4adff24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:33.654734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61eb76-0bfa-4789-8f87-012d4adff24d admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:33.655981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61eb76-0bfa-4789-8f87-012d4adff24d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:33.659356 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61eb76-0bfa-4789-8f87-012d4adff24d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:33.675577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1347/2661] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:33.680147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61eb76-0bfa-4789-8f87-012d4adff24d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:33.689679 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1f1fb-b4d8-415e-be56-ce39e2a04541 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:33.691605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1f1fb-b4d8-415e-be56-ce39e2a04541 service swift] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:33.694237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61eb76-0bfa-4789-8f87-012d4adff24d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:33.701503 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1b344-6e48-450c-b3fe-588d5903578e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:33.701996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1315/2662] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jul 21 09:47:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 09:47:33.703669 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1b344-6e48-450c-b3fe-588d5903578e admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:33.704162 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1b344-6e48-450c-b3fe-588d5903578e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:33.704439 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1b344-6e48-450c-b3fe-588d5903578e admin admin] PATH_INFO: `/v3/projects/e66e8ae7ccad4020a714b13069476142/users/df310987375542fcb28fee1ca7e74bb2/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:33.727408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e1b344-6e48-450c-b3fe-588d5903578e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e66e8ae7ccad4020a714b13069476142, user_id=df310987375542fcb28fee1ca7e74bb2, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:33.730476 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e1b344-6e48-450c-b3fe-588d5903578e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:33.749765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1348/2663] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:33 2022] PUT /identity/v3/projects/e66e8ae7ccad4020a714b13069476142/users/df310987375542fcb28fee1ca7e74bb2/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:33.762740 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb752050-f0cc-4b7f-a862-f6ad6cbf1c0c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:33.763092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb752050-f0cc-4b7f-a862-f6ad6cbf1c0c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:33.763722 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb752050-f0cc-4b7f-a862-f6ad6cbf1c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:33.802851 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-fb752050-f0cc-4b7f-a862-f6ad6cbf1c0c None None] MFA Rules not processed for user `df310987375542fcb28fee1ca7e74bb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:33.804222 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fb752050-f0cc-4b7f-a862-f6ad6cbf1c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:33.897971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:33.944086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1316/2664] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:33.945618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1b344-6e48-450c-b3fe-588d5903578e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:34.100846 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1b344-6e48-450c-b3fe-588d5903578e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:34.105934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357da87-e0d9-4cd1-b81b-d4a96402019e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:34.106605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357da87-e0d9-4cd1-b81b-d4a96402019e admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:34.107488 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357da87-e0d9-4cd1-b81b-d4a96402019e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:34.107754 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357da87-e0d9-4cd1-b81b-d4a96402019e admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:34.109039 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357da87-e0d9-4cd1-b81b-d4a96402019e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:34.112964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357da87-e0d9-4cd1-b81b-d4a96402019e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:34.167973 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1349/2665] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:47:33 2022] POST /identity/v3/projects => generated 375 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:47:34.173154 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb752050-f0cc-4b7f-a862-f6ad6cbf1c0c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:34.192547 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb752050-f0cc-4b7f-a862-f6ad6cbf1c0c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:34.278217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016e687-143a-4092-a504-73d4671e9d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:34.280674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016e687-143a-4092-a504-73d4671e9d5c admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:34.281255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016e687-143a-4092-a504-73d4671e9d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:34.282013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016e687-143a-4092-a504-73d4671e9d5c admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:34.284278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016e687-143a-4092-a504-73d4671e9d5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:34.287766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016e687-143a-4092-a504-73d4671e9d5c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:34.330682 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1317/2666] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:47:34 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:34.332654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357da87-e0d9-4cd1-b81b-d4a96402019e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:34.347239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357da87-e0d9-4cd1-b81b-d4a96402019e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:34.353181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e752e44-be6d-48fc-bf7a-7792f5ae8277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:34.353919 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e752e44-be6d-48fc-bf7a-7792f5ae8277 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:34.354213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e752e44-be6d-48fc-bf7a-7792f5ae8277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:34.354496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e752e44-be6d-48fc-bf7a-7792f5ae8277 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:34.355235 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e752e44-be6d-48fc-bf7a-7792f5ae8277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:34.356901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e752e44-be6d-48fc-bf7a-7792f5ae8277 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:34.372025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1350/2667] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:34.375384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016e687-143a-4092-a504-73d4671e9d5c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:34.388032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016e687-143a-4092-a504-73d4671e9d5c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:34.395696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654952e-4b35-44f6-a60c-a73f284fd542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:34.396756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654952e-4b35-44f6-a60c-a73f284fd542 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:34.397161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654952e-4b35-44f6-a60c-a73f284fd542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:34.397544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654952e-4b35-44f6-a60c-a73f284fd542 admin admin] PATH_INFO: `/v3/projects/d22d29f9a0b8495b9efe310857557900/users/e7a77b67de6a4e4cba8d9487a84fd27c/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:34.418149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654952e-4b35-44f6-a60c-a73f284fd542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d22d29f9a0b8495b9efe310857557900, user_id=e7a77b67de6a4e4cba8d9487a84fd27c, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:34.422337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654952e-4b35-44f6-a60c-a73f284fd542 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:34.445395 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1318/2668] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:34 2022] PUT /identity/v3/projects/d22d29f9a0b8495b9efe310857557900/users/e7a77b67de6a4e4cba8d9487a84fd27c/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:47:34.447048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e752e44-be6d-48fc-bf7a-7792f5ae8277 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:34.458286 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e752e44-be6d-48fc-bf7a-7792f5ae8277 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:34.548273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91520afd-cf0c-47fb-8a37-a2ce9e43896d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:34.549361 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91520afd-cf0c-47fb-8a37-a2ce9e43896d admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:34.549869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91520afd-cf0c-47fb-8a37-a2ce9e43896d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:34.550383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91520afd-cf0c-47fb-8a37-a2ce9e43896d admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:34.551343 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91520afd-cf0c-47fb-8a37-a2ce9e43896d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:34.553508 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91520afd-cf0c-47fb-8a37-a2ce9e43896d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:34.569435 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1351/2669] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:34 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:34.570279 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654952e-4b35-44f6-a60c-a73f284fd542 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:34.584431 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654952e-4b35-44f6-a60c-a73f284fd542 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:34.590802 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7c311-4dfd-4dc5-af05-d574b7390c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:34.593817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7c311-4dfd-4dc5-af05-d574b7390c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:34.594585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7c311-4dfd-4dc5-af05-d574b7390c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:34.595397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7c311-4dfd-4dc5-af05-d574b7390c20 admin admin] PATH_INFO: `/v3/projects/d22d29f9a0b8495b9efe310857557900/users/e7a77b67de6a4e4cba8d9487a84fd27c/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:34.607935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7c311-4dfd-4dc5-af05-d574b7390c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d22d29f9a0b8495b9efe310857557900, user_id=e7a77b67de6a4e4cba8d9487a84fd27c, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:34.611541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7c311-4dfd-4dc5-af05-d574b7390c20 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:34.623878 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1319/2670] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:34 2022] PUT /identity/v3/projects/d22d29f9a0b8495b9efe310857557900/users/e7a77b67de6a4e4cba8d9487a84fd27c/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:34.634087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9aa50-606e-4f18-bfb3-58644275b015 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:34.634404 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9aa50-606e-4f18-bfb3-58644275b015 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:34.634658 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9aa50-606e-4f18-bfb3-58644275b015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:34.664631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-84c9aa50-606e-4f18-bfb3-58644275b015 None None] MFA Rules not processed for user `e7a77b67de6a4e4cba8d9487a84fd27c`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:34.665873 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-84c9aa50-606e-4f18-bfb3-58644275b015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:34.704669 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:34.791751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1352/2671] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:34.803669 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7c311-4dfd-4dc5-af05-d574b7390c20 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:34.806713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-57d7c311-4dfd-4dc5-af05-d574b7390c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:34.941630 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7c311-4dfd-4dc5-af05-d574b7390c20 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:34.943988 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3ed02-99d1-41e3-9b31-dd7e50831282 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:34.944797 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3ed02-99d1-41e3-9b31-dd7e50831282 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:34.945186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3ed02-99d1-41e3-9b31-dd7e50831282 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:34.945536 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3ed02-99d1-41e3-9b31-dd7e50831282 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:35.075186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:35.084778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3ed02-99d1-41e3-9b31-dd7e50831282 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:35.086704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3ed02-99d1-41e3-9b31-dd7e50831282 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:35.112271 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1320/2672] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:36.360172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9aa50-606e-4f18-bfb3-58644275b015 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:36.362163 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-84c9aa50-606e-4f18-bfb3-58644275b015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:36.588613 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9aa50-606e-4f18-bfb3-58644275b015 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:36.605558 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e36fcb-8793-4a6b-8c4e-12bacbe36754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:36.606545 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e36fcb-8793-4a6b-8c4e-12bacbe36754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:36.606838 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e36fcb-8793-4a6b-8c4e-12bacbe36754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:36.607119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e36fcb-8793-4a6b-8c4e-12bacbe36754 admin admin] PATH_INFO: `/v3/users/d23a2825a06e4297a96ab098af99cd5c` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:36.609561 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e36fcb-8793-4a6b-8c4e-12bacbe36754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d23a2825a06e4297a96ab098af99cd5c)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:36.611567 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e36fcb-8793-4a6b-8c4e-12bacbe36754 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:36.702745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-62e36fcb-8793-4a6b-8c4e-12bacbe36754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd23a2825a06e4297a96ab098af99cd5c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:36.707750 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-62e36fcb-8793-4a6b-8c4e-12bacbe36754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd23a2825a06e4297a96ab098af99cd5c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:36.733133 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-62e36fcb-8793-4a6b-8c4e-12bacbe36754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd23a2825a06e4297a96ab098af99cd5c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:36.764891 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-62e36fcb-8793-4a6b-8c4e-12bacbe36754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd23a2825a06e4297a96ab098af99cd5c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:36.782281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-62e36fcb-8793-4a6b-8c4e-12bacbe36754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd23a2825a06e4297a96ab098af99cd5c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:36.791765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1353/2673] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:47:36 2022] DELETE /identity/v3/users/d23a2825a06e4297a96ab098af99cd5c => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:36.798231 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501d4d2-57d5-4034-9ff8-c2b448e45cab None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:36.798688 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501d4d2-57d5-4034-9ff8-c2b448e45cab None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:36.798794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501d4d2-57d5-4034-9ff8-c2b448e45cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:36.822270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c501d4d2-57d5-4034-9ff8-c2b448e45cab None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:36.825153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c501d4d2-57d5-4034-9ff8-c2b448e45cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:36.959163 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1321/2674] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:36.969618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e36fcb-8793-4a6b-8c4e-12bacbe36754 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:36.971131 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-62e36fcb-8793-4a6b-8c4e-12bacbe36754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:37.090642 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:37.124796 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e36fcb-8793-4a6b-8c4e-12bacbe36754 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:37.137993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cbeb70-233f-4b11-b2e3-90efd7e64c52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:37.138782 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbeb70-233f-4b11-b2e3-90efd7e64c52 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:37.139136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbeb70-233f-4b11-b2e3-90efd7e64c52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:37.139485 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cbeb70-233f-4b11-b2e3-90efd7e64c52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:37.158695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cbeb70-233f-4b11-b2e3-90efd7e64c52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:37.160514 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cbeb70-233f-4b11-b2e3-90efd7e64c52 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:37.184752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1354/2675] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:37.495901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501d4d2-57d5-4034-9ff8-c2b448e45cab None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:37.497486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c501d4d2-57d5-4034-9ff8-c2b448e45cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:37.519774 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501d4d2-57d5-4034-9ff8-c2b448e45cab None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:37.525970 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b6142-e4f2-49b6-b2dc-cae0d1ad7179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:37.526855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b6142-e4f2-49b6-b2dc-cae0d1ad7179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:37.527164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b6142-e4f2-49b6-b2dc-cae0d1ad7179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:37.527420 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b6142-e4f2-49b6-b2dc-cae0d1ad7179 admin admin] PATH_INFO: `/v3/projects/63584d9f32ac4da0835773fec871e121` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:37.528838 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b6142-e4f2-49b6-b2dc-cae0d1ad7179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63584d9f32ac4da0835773fec871e121)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:37.531715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b6142-e4f2-49b6-b2dc-cae0d1ad7179 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:37.628694 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d38b6142-e4f2-49b6-b2dc-cae0d1ad7179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63584d9f32ac4da0835773fec871e121', 'request_id': 'req-d38b6142-e4f2-49b6-b2dc-cae0d1ad7179'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:37.648331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d38b6142-e4f2-49b6-b2dc-cae0d1ad7179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63584d9f32ac4da0835773fec871e121', 'request_id': 'req-d38b6142-e4f2-49b6-b2dc-cae0d1ad7179'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:37.669670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d38b6142-e4f2-49b6-b2dc-cae0d1ad7179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63584d9f32ac4da0835773fec871e121', 'request_id': 'req-d38b6142-e4f2-49b6-b2dc-cae0d1ad7179'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:37.687842 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d38b6142-e4f2-49b6-b2dc-cae0d1ad7179 admin admin] The token cache is being invalidate because project 63584d9f32ac4da0835773fec871e121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:47:37.688261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d38b6142-e4f2-49b6-b2dc-cae0d1ad7179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:37.688840 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d38b6142-e4f2-49b6-b2dc-cae0d1ad7179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:37.696524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111d6c6-55cd-432a-93a3-d0739f254d60 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:37.697029 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111d6c6-55cd-432a-93a3-d0739f254d60 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:37.697147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111d6c6-55cd-432a-93a3-d0739f254d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:37.701368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1322/2676] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:47:37 2022] DELETE /identity/v3/projects/63584d9f32ac4da0835773fec871e121 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:37.766185 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-8111d6c6-55cd-432a-93a3-d0739f254d60 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:37.768981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8111d6c6-55cd-432a-93a3-d0739f254d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:38.033963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1355/2677] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:47:38.041947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dabf3-2a36-4ea6-ba58-be70d2a3dee6 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:38.042523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dabf3-2a36-4ea6-ba58-be70d2a3dee6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:38.042523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dabf3-2a36-4ea6-ba58-be70d2a3dee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:38.107762 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ca5dabf3-2a36-4ea6-ba58-be70d2a3dee6 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:38.122505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ca5dabf3-2a36-4ea6-ba58-be70d2a3dee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:38.172487 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1323/2678] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:47:38.178202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111d6c6-55cd-432a-93a3-d0739f254d60 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:38.196420 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111d6c6-55cd-432a-93a3-d0739f254d60 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:38.205540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e598dd-b0c0-4b9d-94db-bdd310eff3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:38.206247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e598dd-b0c0-4b9d-94db-bdd310eff3bd admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:38.206537 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e598dd-b0c0-4b9d-94db-bdd310eff3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:38.206819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e598dd-b0c0-4b9d-94db-bdd310eff3bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:38.207980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e598dd-b0c0-4b9d-94db-bdd310eff3bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:47:38.208221 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e598dd-b0c0-4b9d-94db-bdd310eff3bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:38.210013 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e598dd-b0c0-4b9d-94db-bdd310eff3bd admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:38.234083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1356/2679] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:47:38.238349 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5dabf3-2a36-4ea6-ba58-be70d2a3dee6 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:38.254506 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5dabf3-2a36-4ea6-ba58-be70d2a3dee6 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:38.259875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ef3c1-5c56-4393-bc63-762b9e45d29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:38.260586 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ef3c1-5c56-4393-bc63-762b9e45d29a admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:38.260870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ef3c1-5c56-4393-bc63-762b9e45d29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:38.261149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ef3c1-5c56-4393-bc63-762b9e45d29a admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:38.261860 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ef3c1-5c56-4393-bc63-762b9e45d29a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:38.263979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ef3c1-5c56-4393-bc63-762b9e45d29a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:38.332947 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1324/2680] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:47:38 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:38.337376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e598dd-b0c0-4b9d-94db-bdd310eff3bd admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:38.350845 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e598dd-b0c0-4b9d-94db-bdd310eff3bd admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:38.539898 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edb6db-8fc6-4d35-9a29-bcf22f2d2678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:38.540935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edb6db-8fc6-4d35-9a29-bcf22f2d2678 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:38.541324 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edb6db-8fc6-4d35-9a29-bcf22f2d2678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:38.541708 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edb6db-8fc6-4d35-9a29-bcf22f2d2678 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:38.542530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08edb6db-8fc6-4d35-9a29-bcf22f2d2678 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:38.545190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08edb6db-8fc6-4d35-9a29-bcf22f2d2678 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:38.584990 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1357/2681] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:47:38 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:38.589281 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ef3c1-5c56-4393-bc63-762b9e45d29a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:38.609669 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ef3c1-5c56-4393-bc63-762b9e45d29a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:38.623532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91d8cd-6099-4673-9543-43455a9ea686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:38.625213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91d8cd-6099-4673-9543-43455a9ea686 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:38.626004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91d8cd-6099-4673-9543-43455a9ea686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:38.627361 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91d8cd-6099-4673-9543-43455a9ea686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:38.628323 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d91d8cd-6099-4673-9543-43455a9ea686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:38.630108 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d91d8cd-6099-4673-9543-43455a9ea686 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:38.654133 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1325/2682] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:38.658808 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edb6db-8fc6-4d35-9a29-bcf22f2d2678 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:38.670017 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edb6db-8fc6-4d35-9a29-bcf22f2d2678 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:38.676028 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff11fe4-cfaa-4473-bad1-aae3ef2d5daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:38.676709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff11fe4-cfaa-4473-bad1-aae3ef2d5daf admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:38.676998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff11fe4-cfaa-4473-bad1-aae3ef2d5daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:38.677281 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff11fe4-cfaa-4473-bad1-aae3ef2d5daf admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:38.678149 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff11fe4-cfaa-4473-bad1-aae3ef2d5daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:38.679799 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff11fe4-cfaa-4473-bad1-aae3ef2d5daf admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:38.694758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1358/2683] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:47:38.698799 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91d8cd-6099-4673-9543-43455a9ea686 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:38.710982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91d8cd-6099-4673-9543-43455a9ea686 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:38.723839 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77ab5d-7419-4e26-a3c4-9c1cb04f25a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:38.726541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77ab5d-7419-4e26-a3c4-9c1cb04f25a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:38.728126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77ab5d-7419-4e26-a3c4-9c1cb04f25a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:38.728126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77ab5d-7419-4e26-a3c4-9c1cb04f25a3 admin admin] PATH_INFO: `/v3/projects/253b656640304948a74e7bcc33b54561/users/70eea309b21d4e628d14991a68ceeeef/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:38.748631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77ab5d-7419-4e26-a3c4-9c1cb04f25a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=253b656640304948a74e7bcc33b54561, user_id=70eea309b21d4e628d14991a68ceeeef, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:38.751694 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77ab5d-7419-4e26-a3c4-9c1cb04f25a3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:38.777005 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1326/2684] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:38 2022] PUT /identity/v3/projects/253b656640304948a74e7bcc33b54561/users/70eea309b21d4e628d14991a68ceeeef/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:38.782711 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00303a63-ae9e-4881-b268-39671161435a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:38.783640 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00303a63-ae9e-4881-b268-39671161435a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:38.783961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00303a63-ae9e-4881-b268-39671161435a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:38.821714 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-00303a63-ae9e-4881-b268-39671161435a None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:38.824027 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-00303a63-ae9e-4881-b268-39671161435a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:38.891897 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:38.921682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1359/2685] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:38.932457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77ab5d-7419-4e26-a3c4-9c1cb04f25a3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:38.936768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-da77ab5d-7419-4e26-a3c4-9c1cb04f25a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:38.957132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:39.020489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726ae61-d67c-4434-8946-844c108911b6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:39.021007 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726ae61-d67c-4434-8946-844c108911b6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:39.021450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726ae61-d67c-4434-8946-844c108911b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:39.022943 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77ab5d-7419-4e26-a3c4-9c1cb04f25a3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:39.028115 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351af766-839f-4bad-9f0d-4fa56ec3fcb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:39.028787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351af766-839f-4bad-9f0d-4fa56ec3fcb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:39.030244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351af766-839f-4bad-9f0d-4fa56ec3fcb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:39.031260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351af766-839f-4bad-9f0d-4fa56ec3fcb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:39.042881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351af766-839f-4bad-9f0d-4fa56ec3fcb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:39.045514 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351af766-839f-4bad-9f0d-4fa56ec3fcb6 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:39.064741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1327/2686] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:39.065532 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3726ae61-d67c-4434-8946-844c108911b6 None None] MFA Rules not processed for user `86830104a3524a30aea4d5c5ecc241dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:39.066812 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3726ae61-d67c-4434-8946-844c108911b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:39.166762 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351af766-839f-4bad-9f0d-4fa56ec3fcb6 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:39.167756 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-351af766-839f-4bad-9f0d-4fa56ec3fcb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:39.190064 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351af766-839f-4bad-9f0d-4fa56ec3fcb6 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:39.206948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef933f1-b3dc-47b7-a8dc-f2e75e10fddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:39.208091 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef933f1-b3dc-47b7-a8dc-f2e75e10fddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:39.208377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef933f1-b3dc-47b7-a8dc-f2e75e10fddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:39.208651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef933f1-b3dc-47b7-a8dc-f2e75e10fddd admin admin] PATH_INFO: `/v3/users/df310987375542fcb28fee1ca7e74bb2` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:39.209010 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1360/2687] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:47:39.210416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef933f1-b3dc-47b7-a8dc-f2e75e10fddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=df310987375542fcb28fee1ca7e74bb2)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:39.212685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef933f1-b3dc-47b7-a8dc-f2e75e10fddd admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:39.242825 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726ae61-d67c-4434-8946-844c108911b6 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:39.246451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3726ae61-d67c-4434-8946-844c108911b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:39.277258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cef933f1-b3dc-47b7-a8dc-f2e75e10fddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df310987375542fcb28fee1ca7e74bb2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:39.293066 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cef933f1-b3dc-47b7-a8dc-f2e75e10fddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df310987375542fcb28fee1ca7e74bb2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:39.306676 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cef933f1-b3dc-47b7-a8dc-f2e75e10fddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df310987375542fcb28fee1ca7e74bb2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:39.325060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cef933f1-b3dc-47b7-a8dc-f2e75e10fddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df310987375542fcb28fee1ca7e74bb2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:39.326264 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cef933f1-b3dc-47b7-a8dc-f2e75e10fddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df310987375542fcb28fee1ca7e74bb2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:39.337303 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1328/2688] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:47:39 2022] DELETE /identity/v3/users/df310987375542fcb28fee1ca7e74bb2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:39.348831 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b224bda-8f16-495b-bb6e-c934fb3f855b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:39.351176 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b224bda-8f16-495b-bb6e-c934fb3f855b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:39.353028 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b224bda-8f16-495b-bb6e-c934fb3f855b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:39.398685 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3b224bda-8f16-495b-bb6e-c934fb3f855b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:39.400253 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3b224bda-8f16-495b-bb6e-c934fb3f855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:39.416773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726ae61-d67c-4434-8946-844c108911b6 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:39.488292 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1329/2689] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:47:39.502470 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b224bda-8f16-495b-bb6e-c934fb3f855b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:39.509268 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3b224bda-8f16-495b-bb6e-c934fb3f855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:39.518449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b867a5-cfcf-4ffb-9bea-19c63bcef548 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:39.519788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b867a5-cfcf-4ffb-9bea-19c63bcef548 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:39.520211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b867a5-cfcf-4ffb-9bea-19c63bcef548 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:39.521299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b867a5-cfcf-4ffb-9bea-19c63bcef548 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:39.523593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-49b867a5-cfcf-4ffb-9bea-19c63bcef548 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:39.542287 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:39.624621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:39.638748 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b867a5-cfcf-4ffb-9bea-19c63bcef548 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:39.642254 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b867a5-cfcf-4ffb-9bea-19c63bcef548 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:39.658946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1361/2690] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:39.736151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b224bda-8f16-495b-bb6e-c934fb3f855b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:39.747226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c512b-d46c-4901-a138-960be41f9f52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:39.747226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c512b-d46c-4901-a138-960be41f9f52 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:39.747226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c512b-d46c-4901-a138-960be41f9f52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:39.747226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c512b-d46c-4901-a138-960be41f9f52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:39.774405 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c512b-d46c-4901-a138-960be41f9f52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:39.775997 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c512b-d46c-4901-a138-960be41f9f52 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:39.797143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1330/2691] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:40.463874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b867a5-cfcf-4ffb-9bea-19c63bcef548 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:40.466544 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-49b867a5-cfcf-4ffb-9bea-19c63bcef548 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:40.486609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b867a5-cfcf-4ffb-9bea-19c63bcef548 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:40.493783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:40.496684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:40.496684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:40.496684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5 admin admin] PATH_INFO: `/v3/projects/e66e8ae7ccad4020a714b13069476142` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:40.498086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e66e8ae7ccad4020a714b13069476142)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:40.501498 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:40.589408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e66e8ae7ccad4020a714b13069476142', 'request_id': 'req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:40.607143 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e66e8ae7ccad4020a714b13069476142', 'request_id': 'req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:40.615191 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e66e8ae7ccad4020a714b13069476142', 'request_id': 'req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:40.637827 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5 admin admin] The token cache is being invalidate because project e66e8ae7ccad4020a714b13069476142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:47:40.638931 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:40.651669 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:40.654579 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1362/2692] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:47:40 2022] DELETE /identity/v3/projects/e66e8ae7ccad4020a714b13069476142 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:40.659247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c512b-d46c-4901-a138-960be41f9f52 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:40.673077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c512b-d46c-4901-a138-960be41f9f52 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:40.781962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8400d-e6a8-4f61-927c-4feb275249dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:40.784262 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8400d-e6a8-4f61-927c-4feb275249dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:40.785068 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8400d-e6a8-4f61-927c-4feb275249dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:40.785582 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8400d-e6a8-4f61-927c-4feb275249dd admin admin] PATH_INFO: `/v3/users/e7a77b67de6a4e4cba8d9487a84fd27c` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:40.788173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8400d-e6a8-4f61-927c-4feb275249dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7a77b67de6a4e4cba8d9487a84fd27c)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:40.793056 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8400d-e6a8-4f61-927c-4feb275249dd admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:40.894942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4bb8400d-e6a8-4f61-927c-4feb275249dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7a77b67de6a4e4cba8d9487a84fd27c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:40.917425 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4bb8400d-e6a8-4f61-927c-4feb275249dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7a77b67de6a4e4cba8d9487a84fd27c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:40.933660 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4bb8400d-e6a8-4f61-927c-4feb275249dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7a77b67de6a4e4cba8d9487a84fd27c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:40.959819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4bb8400d-e6a8-4f61-927c-4feb275249dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7a77b67de6a4e4cba8d9487a84fd27c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:40.962543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4bb8400d-e6a8-4f61-927c-4feb275249dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7a77b67de6a4e4cba8d9487a84fd27c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:40.975991 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1331/2693] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:47:40 2022] DELETE /identity/v3/users/e7a77b67de6a4e4cba8d9487a84fd27c => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:47:41.576203 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:41.578430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:41.600530 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe6d4f-a9b9-4a0b-9213-f9780d07cfd5 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:41.757620 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21de2c-1201-48e0-b543-1ff803654b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:41.758756 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21de2c-1201-48e0-b543-1ff803654b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:41.759044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21de2c-1201-48e0-b543-1ff803654b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:41.759326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21de2c-1201-48e0-b543-1ff803654b4b admin admin] PATH_INFO: `/v3/projects/d22d29f9a0b8495b9efe310857557900` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:41.767993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21de2c-1201-48e0-b543-1ff803654b4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d22d29f9a0b8495b9efe310857557900)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:41.767993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21de2c-1201-48e0-b543-1ff803654b4b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:41.776420 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad846ab9-5420-4ce8-af30-d6d534ee1fb9 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:41.776420 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad846ab9-5420-4ce8-af30-d6d534ee1fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:41.776420 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad846ab9-5420-4ce8-af30-d6d534ee1fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:41.795494 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ad846ab9-5420-4ce8-af30-d6d534ee1fb9 None None] MFA Rules not processed for user `9d439ed3cd1f4610a4d4902b42d19f37`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:41.798967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ad846ab9-5420-4ce8-af30-d6d534ee1fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:41.886011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:41.889997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:41.917274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8a21de2c-1201-48e0-b543-1ff803654b4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd22d29f9a0b8495b9efe310857557900', 'request_id': 'req-8a21de2c-1201-48e0-b543-1ff803654b4b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:41.928511 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1332/2694] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:41.938751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8a21de2c-1201-48e0-b543-1ff803654b4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd22d29f9a0b8495b9efe310857557900', 'request_id': 'req-8a21de2c-1201-48e0-b543-1ff803654b4b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:41.953148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8a21de2c-1201-48e0-b543-1ff803654b4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd22d29f9a0b8495b9efe310857557900', 'request_id': 'req-8a21de2c-1201-48e0-b543-1ff803654b4b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:41.986921 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8a21de2c-1201-48e0-b543-1ff803654b4b admin admin] The token cache is being invalidate because project d22d29f9a0b8495b9efe310857557900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:47:41.987606 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8a21de2c-1201-48e0-b543-1ff803654b4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:41.992152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8a21de2c-1201-48e0-b543-1ff803654b4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:41.996225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1363/2695] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:47:41 2022] DELETE /identity/v3/projects/d22d29f9a0b8495b9efe310857557900 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:47:42.002328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a299cc7-18a0-435b-a09f-577d884b1981 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:42.002754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a299cc7-18a0-435b-a09f-577d884b1981 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:42.003197 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a299cc7-18a0-435b-a09f-577d884b1981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:42.054637 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2a299cc7-18a0-435b-a09f-577d884b1981 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:42.057158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2a299cc7-18a0-435b-a09f-577d884b1981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:42.162702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1333/2696] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:42.177081 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c57189-6cb5-441b-99e9-f9acf232214e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:42.177421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c57189-6cb5-441b-99e9-f9acf232214e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:42.178350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c57189-6cb5-441b-99e9-f9acf232214e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:42.232329 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f1c57189-6cb5-441b-99e9-f9acf232214e None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:42.235124 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f1c57189-6cb5-441b-99e9-f9acf232214e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:42.252210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1364/2697] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:42.259190 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a299cc7-18a0-435b-a09f-577d884b1981 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:42.272514 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a299cc7-18a0-435b-a09f-577d884b1981 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:42.278430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d5972-4da2-43ba-a74c-24569cefcd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:42.278948 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d5972-4da2-43ba-a74c-24569cefcd1d admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:42.279204 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d5972-4da2-43ba-a74c-24569cefcd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:42.279830 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d5972-4da2-43ba-a74c-24569cefcd1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:42.280359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d5972-4da2-43ba-a74c-24569cefcd1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:47:42.280675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d5972-4da2-43ba-a74c-24569cefcd1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:42.282521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d5972-4da2-43ba-a74c-24569cefcd1d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:42.314983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1334/2698] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:47:42.322478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c57189-6cb5-441b-99e9-f9acf232214e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:42.368935 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c57189-6cb5-441b-99e9-f9acf232214e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:42.378906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547155a-4cae-4364-a450-719170107886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:42.381059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547155a-4cae-4364-a450-719170107886 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:42.381059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547155a-4cae-4364-a450-719170107886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:42.381469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547155a-4cae-4364-a450-719170107886 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:42.382264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547155a-4cae-4364-a450-719170107886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:42.392433 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547155a-4cae-4364-a450-719170107886 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:42.469773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1365/2699] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:47:42 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:42.470461 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d5972-4da2-43ba-a74c-24569cefcd1d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:42.491984 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d5972-4da2-43ba-a74c-24569cefcd1d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:42.644849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86795a82-abcd-4845-a6d7-9153b71d24cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:42.646476 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86795a82-abcd-4845-a6d7-9153b71d24cf admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:42.646983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86795a82-abcd-4845-a6d7-9153b71d24cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:42.647601 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86795a82-abcd-4845-a6d7-9153b71d24cf admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:42.649737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86795a82-abcd-4845-a6d7-9153b71d24cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:42.652164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86795a82-abcd-4845-a6d7-9153b71d24cf admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:42.744356 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1335/2700] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:47:42 2022] POST /identity/v3/users => generated 288 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:47:42.745370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547155a-4cae-4364-a450-719170107886 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:42.767088 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547155a-4cae-4364-a450-719170107886 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:42.794961 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d585d-3b06-406c-91c7-c544294b522c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:42.796405 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d585d-3b06-406c-91c7-c544294b522c admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:42.796907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d585d-3b06-406c-91c7-c544294b522c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:42.802031 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d585d-3b06-406c-91c7-c544294b522c admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:42.803308 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d585d-3b06-406c-91c7-c544294b522c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:42.808579 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d585d-3b06-406c-91c7-c544294b522c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:42.832883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1366/2701] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:47:42.837737 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86795a82-abcd-4845-a6d7-9153b71d24cf admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:42.859717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86795a82-abcd-4845-a6d7-9153b71d24cf admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:42.881331 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696a2ff-21c4-440b-9b97-c8a7be2802f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:42.884577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696a2ff-21c4-440b-9b97-c8a7be2802f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:42.885013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696a2ff-21c4-440b-9b97-c8a7be2802f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:42.885478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696a2ff-21c4-440b-9b97-c8a7be2802f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:42.886297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696a2ff-21c4-440b-9b97-c8a7be2802f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:42.888781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696a2ff-21c4-440b-9b97-c8a7be2802f7 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:42.908093 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1336/2702] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:42 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:42.912314 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d585d-3b06-406c-91c7-c544294b522c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:42.927026 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d585d-3b06-406c-91c7-c544294b522c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:42.948493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef5845-49c1-4665-88f7-3139dd47feee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:42.951870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef5845-49c1-4665-88f7-3139dd47feee admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:42.955668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef5845-49c1-4665-88f7-3139dd47feee admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:42.955993 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef5845-49c1-4665-88f7-3139dd47feee admin admin] PATH_INFO: `/v3/projects/ddb11566bf7b45568f0da1d0be7f9132/users/629e5cf893c44de4aa3f9d303e046f8e/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:42.994666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef5845-49c1-4665-88f7-3139dd47feee admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb11566bf7b45568f0da1d0be7f9132, user_id=629e5cf893c44de4aa3f9d303e046f8e, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:42.997633 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef5845-49c1-4665-88f7-3139dd47feee admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:43.020750 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1367/2703] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:42 2022] PUT /identity/v3/projects/ddb11566bf7b45568f0da1d0be7f9132/users/629e5cf893c44de4aa3f9d303e046f8e/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:43.031851 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934187e6-c5a3-42fa-a349-ef61c75b4833 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:43.032453 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934187e6-c5a3-42fa-a349-ef61c75b4833 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:43.033232 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934187e6-c5a3-42fa-a349-ef61c75b4833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:43.084835 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-934187e6-c5a3-42fa-a349-ef61c75b4833 None None] MFA Rules not processed for user `629e5cf893c44de4aa3f9d303e046f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:43.086158 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-934187e6-c5a3-42fa-a349-ef61c75b4833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:43.161697 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:43.198278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1337/2704] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:43.204574 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef5845-49c1-4665-88f7-3139dd47feee admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:43.324355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef5845-49c1-4665-88f7-3139dd47feee admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:43.329391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617a7db-11a7-4ba8-a240-87cafe5f9b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:43.330078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617a7db-11a7-4ba8-a240-87cafe5f9b02 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:43.330415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617a7db-11a7-4ba8-a240-87cafe5f9b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:43.331038 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617a7db-11a7-4ba8-a240-87cafe5f9b02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:43.331763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617a7db-11a7-4ba8-a240-87cafe5f9b02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:43.333820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617a7db-11a7-4ba8-a240-87cafe5f9b02 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:43.373017 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1368/2705] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:47:43 2022] POST /identity/v3/projects => generated 375 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:43.377365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934187e6-c5a3-42fa-a349-ef61c75b4833 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:43.397280 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934187e6-c5a3-42fa-a349-ef61c75b4833 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:43.575532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd4bcf-3c8a-4ecc-9f00-151c38f04659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:43.576328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd4bcf-3c8a-4ecc-9f00-151c38f04659 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:43.576618 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd4bcf-3c8a-4ecc-9f00-151c38f04659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:43.576976 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd4bcf-3c8a-4ecc-9f00-151c38f04659 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:43.580518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd4bcf-3c8a-4ecc-9f00-151c38f04659 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:43.583336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd4bcf-3c8a-4ecc-9f00-151c38f04659 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:43.622903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1338/2706] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:47:43 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:43.627713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617a7db-11a7-4ba8-a240-87cafe5f9b02 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:43.651690 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617a7db-11a7-4ba8-a240-87cafe5f9b02 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:43.666583 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6b954-c70f-4f10-8eb5-a27d2aae704f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:43.668385 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6b954-c70f-4f10-8eb5-a27d2aae704f admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:43.668887 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6b954-c70f-4f10-8eb5-a27d2aae704f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:43.669587 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6b954-c70f-4f10-8eb5-a27d2aae704f admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:43.671674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6b954-c70f-4f10-8eb5-a27d2aae704f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:43.682047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6b954-c70f-4f10-8eb5-a27d2aae704f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:43.708425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1369/2707] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:43.712711 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd4bcf-3c8a-4ecc-9f00-151c38f04659 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:43.750430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd4bcf-3c8a-4ecc-9f00-151c38f04659 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:43.757335 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d1b9b-ad00-42dd-ba5a-d21d3e433bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:43.760993 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d1b9b-ad00-42dd-ba5a-d21d3e433bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:43.761681 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d1b9b-ad00-42dd-ba5a-d21d3e433bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:43.762190 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d1b9b-ad00-42dd-ba5a-d21d3e433bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:43.763815 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d1b9b-ad00-42dd-ba5a-d21d3e433bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:43.766922 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d1b9b-ad00-42dd-ba5a-d21d3e433bfb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:43.783806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1339/2708] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:47:43.790129 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6b954-c70f-4f10-8eb5-a27d2aae704f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:43.804509 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6b954-c70f-4f10-8eb5-a27d2aae704f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:43.810500 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9722c3-ae2a-49df-9e8d-f72bcfae0778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:43.812509 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9722c3-ae2a-49df-9e8d-f72bcfae0778 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:43.813037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9722c3-ae2a-49df-9e8d-f72bcfae0778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:43.813556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9722c3-ae2a-49df-9e8d-f72bcfae0778 admin admin] PATH_INFO: `/v3/projects/8007456682094d64aa3082018a763ef6/users/2715f540bfec4ed680c91d4484755409/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:43.873198 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9722c3-ae2a-49df-9e8d-f72bcfae0778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8007456682094d64aa3082018a763ef6, user_id=2715f540bfec4ed680c91d4484755409, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:43.895844 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9722c3-ae2a-49df-9e8d-f72bcfae0778 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:43.939408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1370/2709] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:43 2022] PUT /identity/v3/projects/8007456682094d64aa3082018a763ef6/users/2715f540bfec4ed680c91d4484755409/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:43.962933 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62808feb-aea5-41f4-bbc5-af8945c796b4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:43.963438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62808feb-aea5-41f4-bbc5-af8945c796b4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:43.963564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62808feb-aea5-41f4-bbc5-af8945c796b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:44.031371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-62808feb-aea5-41f4-bbc5-af8945c796b4 None None] MFA Rules not processed for user `2715f540bfec4ed680c91d4484755409`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:44.036218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-62808feb-aea5-41f4-bbc5-af8945c796b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:44.214989 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1340/2710] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:47:44.219987 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9722c3-ae2a-49df-9e8d-f72bcfae0778 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:44.237059 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:44.349524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9722c3-ae2a-49df-9e8d-f72bcfae0778 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:44.356972 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aef86c-523c-4d7f-a229-be54860c39a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:44.357790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aef86c-523c-4d7f-a229-be54860c39a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:44.358702 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aef86c-523c-4d7f-a229-be54860c39a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:44.359213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aef86c-523c-4d7f-a229-be54860c39a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:44.361056 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aef86c-523c-4d7f-a229-be54860c39a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:44.364723 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aef86c-523c-4d7f-a229-be54860c39a5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:44.416838 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1371/2711] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:47:44 2022] POST /identity/v3/projects => generated 375 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:44.421853 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62808feb-aea5-41f4-bbc5-af8945c796b4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:44.445325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62808feb-aea5-41f4-bbc5-af8945c796b4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:44.615678 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b306ac7-4221-4913-8d90-6fd0e567662a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:44.616597 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b306ac7-4221-4913-8d90-6fd0e567662a admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:44.617383 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b306ac7-4221-4913-8d90-6fd0e567662a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:44.617383 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b306ac7-4221-4913-8d90-6fd0e567662a admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:44.621343 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b306ac7-4221-4913-8d90-6fd0e567662a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:44.627693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b306ac7-4221-4913-8d90-6fd0e567662a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:44.665298 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:44.696887 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1341/2712] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:47:44 2022] POST /identity/v3/users => generated 294 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:44.701014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aef86c-523c-4d7f-a229-be54860c39a5 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:44.734527 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aef86c-523c-4d7f-a229-be54860c39a5 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:44.742320 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3815ab-3910-4dd2-b1a7-54ba92c868df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:44.743033 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3815ab-3910-4dd2-b1a7-54ba92c868df admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:44.743321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3815ab-3910-4dd2-b1a7-54ba92c868df admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:44.743603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3815ab-3910-4dd2-b1a7-54ba92c868df admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:44.744430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3815ab-3910-4dd2-b1a7-54ba92c868df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:44.746199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3815ab-3910-4dd2-b1a7-54ba92c868df admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:44.791585 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1372/2713] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:44.796338 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b306ac7-4221-4913-8d90-6fd0e567662a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:44.816652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b306ac7-4221-4913-8d90-6fd0e567662a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:44.832115 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c81ac-697c-4df5-9a1d-691d0160ce66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:44.837658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c81ac-697c-4df5-9a1d-691d0160ce66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:44.839695 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c81ac-697c-4df5-9a1d-691d0160ce66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:44.846069 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c81ac-697c-4df5-9a1d-691d0160ce66 admin admin] PATH_INFO: `/v3/projects/395606b2e96c4b3d9b6c833dd3b8eb3b/users/ef4b6f15ebc24040bda557b6340ef543/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:44.893165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c81ac-697c-4df5-9a1d-691d0160ce66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=395606b2e96c4b3d9b6c833dd3b8eb3b, user_id=ef4b6f15ebc24040bda557b6340ef543, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:44.902955 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c81ac-697c-4df5-9a1d-691d0160ce66 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:44.933260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1342/2714] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:44 2022] PUT /identity/v3/projects/395606b2e96c4b3d9b6c833dd3b8eb3b/users/ef4b6f15ebc24040bda557b6340ef543/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:44.937888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3815ab-3910-4dd2-b1a7-54ba92c868df admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:44.958776 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3815ab-3910-4dd2-b1a7-54ba92c868df admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:45.128976 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ebe557-d98a-4a39-88dc-49d27bd3e2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:45.132470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebe557-d98a-4a39-88dc-49d27bd3e2c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:45.132915 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebe557-d98a-4a39-88dc-49d27bd3e2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:45.133487 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebe557-d98a-4a39-88dc-49d27bd3e2c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:45.134322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ebe557-d98a-4a39-88dc-49d27bd3e2c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:45.136093 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ebe557-d98a-4a39-88dc-49d27bd3e2c8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:45.161579 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1373/2715] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:45.166005 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c81ac-697c-4df5-9a1d-691d0160ce66 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:45.192603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c81ac-697c-4df5-9a1d-691d0160ce66 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:45.199596 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51024318-64d2-430d-9cbf-4f4be13d4d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:45.201155 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024318-64d2-430d-9cbf-4f4be13d4d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:45.201725 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024318-64d2-430d-9cbf-4f4be13d4d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:45.202968 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024318-64d2-430d-9cbf-4f4be13d4d03 admin admin] PATH_INFO: `/v3/projects/395606b2e96c4b3d9b6c833dd3b8eb3b/users/ef4b6f15ebc24040bda557b6340ef543/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:45.209034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51024318-64d2-430d-9cbf-4f4be13d4d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=395606b2e96c4b3d9b6c833dd3b8eb3b, user_id=ef4b6f15ebc24040bda557b6340ef543, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:45.214308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51024318-64d2-430d-9cbf-4f4be13d4d03 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:45.254603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1343/2716] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:45 2022] PUT /identity/v3/projects/395606b2e96c4b3d9b6c833dd3b8eb3b/users/ef4b6f15ebc24040bda557b6340ef543/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:45.268275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea66a64-befe-4140-bd7c-59a3443729c8 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:45.268780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea66a64-befe-4140-bd7c-59a3443729c8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:45.270126 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea66a64-befe-4140-bd7c-59a3443729c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:45.335475 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3ea66a64-befe-4140-bd7c-59a3443729c8 None None] MFA Rules not processed for user `ef4b6f15ebc24040bda557b6340ef543`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:45.336681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3ea66a64-befe-4140-bd7c-59a3443729c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:45.468549 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1374/2717] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:45.480095 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51024318-64d2-430d-9cbf-4f4be13d4d03 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:45.481813 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-51024318-64d2-430d-9cbf-4f4be13d4d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:45.596238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51024318-64d2-430d-9cbf-4f4be13d4d03 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:45.608782 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d77e74-ea85-42ff-9b84-5259b05c29b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:45.609789 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d77e74-ea85-42ff-9b84-5259b05c29b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:45.610120 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d77e74-ea85-42ff-9b84-5259b05c29b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:45.610492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d77e74-ea85-42ff-9b84-5259b05c29b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:45.715511 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d77e74-ea85-42ff-9b84-5259b05c29b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:45.717135 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d77e74-ea85-42ff-9b84-5259b05c29b4 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:45.730130 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1344/2718] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:46.756452 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fb768-158d-41f7-aef2-b6eb6db12ce2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:46.757048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fb768-158d-41f7-aef2-b6eb6db12ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:46.757048 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fb768-158d-41f7-aef2-b6eb6db12ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:46.764253 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:46.791628 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-998fb768-158d-41f7-aef2-b6eb6db12ce2 None None] MFA Rules not processed for user `70eea309b21d4e628d14991a68ceeeef`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:46.792946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-998fb768-158d-41f7-aef2-b6eb6db12ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:46.880091 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1375/2719] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:46.881693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d77e74-ea85-42ff-9b84-5259b05c29b4 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:46.883300 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-83d77e74-ea85-42ff-9b84-5259b05c29b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:46.966804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d77e74-ea85-42ff-9b84-5259b05c29b4 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:46.979570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99d3b9-0a0c-4475-aec2-27abcf5c3793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:46.984381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99d3b9-0a0c-4475-aec2-27abcf5c3793 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:46.985583 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99d3b9-0a0c-4475-aec2-27abcf5c3793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:46.986426 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99d3b9-0a0c-4475-aec2-27abcf5c3793 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:46.988267 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99d3b9-0a0c-4475-aec2-27abcf5c3793 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:46.991896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99d3b9-0a0c-4475-aec2-27abcf5c3793 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:47.010564 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:47.038381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1345/2720] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:47:46 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:47.039659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fb768-158d-41f7-aef2-b6eb6db12ce2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:47.055029 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fb768-158d-41f7-aef2-b6eb6db12ce2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:47.122337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f00116-1144-465c-b4a4-a47582cea537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:47.123030 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f00116-1144-465c-b4a4-a47582cea537 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:47.123324 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f00116-1144-465c-b4a4-a47582cea537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:47.124140 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f00116-1144-465c-b4a4-a47582cea537 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:47.125152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f00116-1144-465c-b4a4-a47582cea537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:47.127296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f00116-1144-465c-b4a4-a47582cea537 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:47.153763 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1376/2721] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:47:47 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:47:47.158940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99d3b9-0a0c-4475-aec2-27abcf5c3793 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:47.172934 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99d3b9-0a0c-4475-aec2-27abcf5c3793 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:47.178034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb6ea7-3b46-4b1e-ad1b-4caec3914c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:47.178732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb6ea7-3b46-4b1e-ad1b-4caec3914c0d admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:47.179019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb6ea7-3b46-4b1e-ad1b-4caec3914c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:47.179297 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb6ea7-3b46-4b1e-ad1b-4caec3914c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:47.180068 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb6ea7-3b46-4b1e-ad1b-4caec3914c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:47.181706 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb6ea7-3b46-4b1e-ad1b-4caec3914c0d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:47.204274 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1346/2722] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:47.204419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f00116-1144-465c-b4a4-a47582cea537 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:47.210269 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f00116-1144-465c-b4a4-a47582cea537 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:47.217110 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ab351-fa63-4b71-8447-3ae9654a9215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:47.218775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ab351-fa63-4b71-8447-3ae9654a9215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:47.218930 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ab351-fa63-4b71-8447-3ae9654a9215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:47.219168 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ab351-fa63-4b71-8447-3ae9654a9215 admin admin] PATH_INFO: `/v3/projects/82561dda7cff4e9eb8cfa6b0a46ed989/users/9cce43e8533e4a6e96800538aa90d8d1/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:47.236970 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ab351-fa63-4b71-8447-3ae9654a9215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82561dda7cff4e9eb8cfa6b0a46ed989, user_id=9cce43e8533e4a6e96800538aa90d8d1, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:47.239718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ab351-fa63-4b71-8447-3ae9654a9215 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:47.263555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1377/2723] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:47 2022] PUT /identity/v3/projects/82561dda7cff4e9eb8cfa6b0a46ed989/users/9cce43e8533e4a6e96800538aa90d8d1/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:47.269247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb6ea7-3b46-4b1e-ad1b-4caec3914c0d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:47.282400 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb6ea7-3b46-4b1e-ad1b-4caec3914c0d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:47.354942 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b0113-2371-40df-ae90-682198260657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:47.356064 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b0113-2371-40df-ae90-682198260657 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:47.356763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b0113-2371-40df-ae90-682198260657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:47.357365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b0113-2371-40df-ae90-682198260657 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:47.358656 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b0113-2371-40df-ae90-682198260657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:47.361152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b0113-2371-40df-ae90-682198260657 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:47.375477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1347/2724] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:47:47.379980 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ab351-fa63-4b71-8447-3ae9654a9215 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:47.392628 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ab351-fa63-4b71-8447-3ae9654a9215 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:47.398246 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee3420-129a-43b7-8e9a-a3b1e59a7764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:47.399178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee3420-129a-43b7-8e9a-a3b1e59a7764 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:47.399462 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee3420-129a-43b7-8e9a-a3b1e59a7764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:47.399739 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee3420-129a-43b7-8e9a-a3b1e59a7764 admin admin] PATH_INFO: `/v3/projects/82561dda7cff4e9eb8cfa6b0a46ed989/users/9cce43e8533e4a6e96800538aa90d8d1/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:47.401952 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee3420-129a-43b7-8e9a-a3b1e59a7764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82561dda7cff4e9eb8cfa6b0a46ed989, user_id=9cce43e8533e4a6e96800538aa90d8d1, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:47.404758 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee3420-129a-43b7-8e9a-a3b1e59a7764 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:47.417458 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1378/2725] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:47:47 2022] PUT /identity/v3/projects/82561dda7cff4e9eb8cfa6b0a46ed989/users/9cce43e8533e4a6e96800538aa90d8d1/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:47:48.889031 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b0113-2371-40df-ae90-682198260657 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:49.002861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b0113-2371-40df-ae90-682198260657 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:49.088379 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48f144-ff6f-4457-a679-bf9fc8fb715b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:49.089186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48f144-ff6f-4457-a679-bf9fc8fb715b service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:49.089477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48f144-ff6f-4457-a679-bf9fc8fb715b service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:49.089761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48f144-ff6f-4457-a679-bf9fc8fb715b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:49.166004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:49.189514 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48f144-ff6f-4457-a679-bf9fc8fb715b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:49.191034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48f144-ff6f-4457-a679-bf9fc8fb715b service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:49.223788 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1348/2726] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:50.442757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee3420-129a-43b7-8e9a-a3b1e59a7764 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:50.481326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee3420-129a-43b7-8e9a-a3b1e59a7764 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:50.499575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31f2c6-4da7-44ba-b73d-566dbeb734d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:50.511946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31f2c6-4da7-44ba-b73d-566dbeb734d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:50.515882 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31f2c6-4da7-44ba-b73d-566dbeb734d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:50.519366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31f2c6-4da7-44ba-b73d-566dbeb734d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:50.761310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:50.834509 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31f2c6-4da7-44ba-b73d-566dbeb734d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:50.839834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31f2c6-4da7-44ba-b73d-566dbeb734d8 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:50.885387 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1379/2727] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:50 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:53.480431 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48f144-ff6f-4457-a679-bf9fc8fb715b service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:53.484583 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bf48f144-ff6f-4457-a679-bf9fc8fb715b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:53.603746 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48f144-ff6f-4457-a679-bf9fc8fb715b service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:53.610407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632e103-8ac4-47a3-bc35-eb353950dee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:53.611328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632e103-8ac4-47a3-bc35-eb353950dee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:53.611608 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632e103-8ac4-47a3-bc35-eb353950dee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:53.611881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632e103-8ac4-47a3-bc35-eb353950dee0 admin admin] PATH_INFO: `/v3/users/6dd9a208282d4a35a444069855b8b1d5` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:53.613490 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632e103-8ac4-47a3-bc35-eb353950dee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dd9a208282d4a35a444069855b8b1d5)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:53.615557 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632e103-8ac4-47a3-bc35-eb353950dee0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:53.694568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5632e103-8ac4-47a3-bc35-eb353950dee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dd9a208282d4a35a444069855b8b1d5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:53.708503 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5632e103-8ac4-47a3-bc35-eb353950dee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dd9a208282d4a35a444069855b8b1d5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:53.725730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5632e103-8ac4-47a3-bc35-eb353950dee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dd9a208282d4a35a444069855b8b1d5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:53.767985 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5632e103-8ac4-47a3-bc35-eb353950dee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dd9a208282d4a35a444069855b8b1d5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:53.768870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5632e103-8ac4-47a3-bc35-eb353950dee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dd9a208282d4a35a444069855b8b1d5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:53.810757 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1349/2728] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:47:53 2022] DELETE /identity/v3/users/6dd9a208282d4a35a444069855b8b1d5 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:53.816750 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe427c6-8de5-4a93-a2e1-381a3f7373ef None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:53.817480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe427c6-8de5-4a93-a2e1-381a3f7373ef None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:53.817480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe427c6-8de5-4a93-a2e1-381a3f7373ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:53.852101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-bbe427c6-8de5-4a93-a2e1-381a3f7373ef None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:53.853431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-bbe427c6-8de5-4a93-a2e1-381a3f7373ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:53.944464 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1380/2729] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:53.957659 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632e103-8ac4-47a3-bc35-eb353950dee0 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:53.959559 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5632e103-8ac4-47a3-bc35-eb353950dee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:54.161993 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632e103-8ac4-47a3-bc35-eb353950dee0 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:54.176652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30ce25-4e81-45a8-826c-5f81329026a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:54.177409 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30ce25-4e81-45a8-826c-5f81329026a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:54.177700 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30ce25-4e81-45a8-826c-5f81329026a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:54.177981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30ce25-4e81-45a8-826c-5f81329026a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:54.210591 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c30ce25-4e81-45a8-826c-5f81329026a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:54.211017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c30ce25-4e81-45a8-826c-5f81329026a1 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:54.241070 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1350/2730] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:54.288669 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd05c44-d567-41ae-b5ff-774099bc2019 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:54.289190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd05c44-d567-41ae-b5ff-774099bc2019 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:54.289705 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd05c44-d567-41ae-b5ff-774099bc2019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:54.335205 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3fd05c44-d567-41ae-b5ff-774099bc2019 None None] MFA Rules not processed for user `9cce43e8533e4a6e96800538aa90d8d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:54.336525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3fd05c44-d567-41ae-b5ff-774099bc2019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:54.469179 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1381/2731] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:54.484198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30ce25-4e81-45a8-826c-5f81329026a1 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:54.486741 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7c30ce25-4e81-45a8-826c-5f81329026a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:54.510763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:54.610407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30ce25-4e81-45a8-826c-5f81329026a1 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:54.617164 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9106515-5849-4ba6-91f5-ba4e831a401b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e053b6b25a4e4a20a05e6ffa6683222b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:54.618262 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9106515-5849-4ba6-91f5-ba4e831a401b service glance] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:54.618769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9106515-5849-4ba6-91f5-ba4e831a401b service glance] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:54.619224 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9106515-5849-4ba6-91f5-ba4e831a401b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:54.622397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e9106515-5849-4ba6-91f5-ba4e831a401b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:54.765183 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd05c44-d567-41ae-b5ff-774099bc2019 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:54.768149 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9106515-5849-4ba6-91f5-ba4e831a401b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:54.768370 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3fd05c44-d567-41ae-b5ff-774099bc2019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:54.769849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9106515-5849-4ba6-91f5-ba4e831a401b service glance] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:54.793659 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1351/2732] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:54.809207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd05c44-d567-41ae-b5ff-774099bc2019 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:54.821870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:54.821870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:54.821870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:54.821870 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd admin admin] PATH_INFO: `/v3/projects/0ff87f1adad5487ab49f7f4ae8859df7` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:54.822586 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ff87f1adad5487ab49f7f4ae8859df7)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:54.825849 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:54.908475 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ff87f1adad5487ab49f7f4ae8859df7', 'request_id': 'req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:54.916376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ff87f1adad5487ab49f7f4ae8859df7', 'request_id': 'req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:54.922835 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:54.934310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ff87f1adad5487ab49f7f4ae8859df7', 'request_id': 'req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:54.973533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a81683-c2a2-47b8-9fc5-c3b9a9155f8f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:54.973533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a81683-c2a2-47b8-9fc5-c3b9a9155f8f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:54.973533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a81683-c2a2-47b8-9fc5-c3b9a9155f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:54.980956 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd admin admin] The token cache is being invalidate because project 0ff87f1adad5487ab49f7f4ae8859df7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:47:54.981724 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:54.984742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:54.988274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1382/2733] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:47:54 2022] DELETE /identity/v3/projects/0ff87f1adad5487ab49f7f4ae8859df7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:47:55.014782 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-29a81683-c2a2-47b8-9fc5-c3b9a9155f8f None None] MFA Rules not processed for user `257450db238d4efda783e30cbcd23d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:55.017333 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-29a81683-c2a2-47b8-9fc5-c3b9a9155f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:55.205215 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1352/2734] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jul 21 09:47:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:55.222817 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:55.224622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:55.343729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e2fc1-8f84-4443-a6e5-b74f45e9ffdd admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:55.349596 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afbdad-9f29-4826-9bb7-e226f6172ebb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061def74ec7f42ce9a174e0944bf499f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:55.350280 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afbdad-9f29-4826-9bb7-e226f6172ebb service swift] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:55.350569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afbdad-9f29-4826-9bb7-e226f6172ebb service swift] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:55.350852 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afbdad-9f29-4826-9bb7-e226f6172ebb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:55.363063 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afbdad-9f29-4826-9bb7-e226f6172ebb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:55.363272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afbdad-9f29-4826-9bb7-e226f6172ebb service swift] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:55.372963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1383/2735] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jul 21 09:47:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 09:47:55.936598 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a81683-c2a2-47b8-9fc5-c3b9a9155f8f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:55.938849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-29a81683-c2a2-47b8-9fc5-c3b9a9155f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:56.119252 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a81683-c2a2-47b8-9fc5-c3b9a9155f8f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:56.130610 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53608ad7-7af0-4e12-b548-51d7db63c85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:56.132226 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53608ad7-7af0-4e12-b548-51d7db63c85e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:56.132771 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53608ad7-7af0-4e12-b548-51d7db63c85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:56.133116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53608ad7-7af0-4e12-b548-51d7db63c85e admin admin] PATH_INFO: `/v3/users/629e5cf893c44de4aa3f9d303e046f8e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:56.135034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53608ad7-7af0-4e12-b548-51d7db63c85e admin admin] RBAC: Authorizing `identity:delete_user(user_id=629e5cf893c44de4aa3f9d303e046f8e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:56.137528 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53608ad7-7af0-4e12-b548-51d7db63c85e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:56.160657 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:56.294230 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-53608ad7-7af0-4e12-b548-51d7db63c85e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '629e5cf893c44de4aa3f9d303e046f8e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:56.308929 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-53608ad7-7af0-4e12-b548-51d7db63c85e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '629e5cf893c44de4aa3f9d303e046f8e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:56.346302 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-53608ad7-7af0-4e12-b548-51d7db63c85e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '629e5cf893c44de4aa3f9d303e046f8e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:56.396215 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-53608ad7-7af0-4e12-b548-51d7db63c85e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '629e5cf893c44de4aa3f9d303e046f8e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:56.397247 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-53608ad7-7af0-4e12-b548-51d7db63c85e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '629e5cf893c44de4aa3f9d303e046f8e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:56.411796 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18563377-1f05-4f69-86bd-a4e405c1c11c None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:56.412571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18563377-1f05-4f69-86bd-a4e405c1c11c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:56.412571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18563377-1f05-4f69-86bd-a4e405c1c11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:56.413820 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1353/2736] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:47:55 2022] DELETE /identity/v3/users/629e5cf893c44de4aa3f9d303e046f8e => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:56.445809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-18563377-1f05-4f69-86bd-a4e405c1c11c None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:47:56.447169 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-18563377-1f05-4f69-86bd-a4e405c1c11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:56.546532 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1384/2737] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:47:56.556869 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53608ad7-7af0-4e12-b548-51d7db63c85e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:56.558355 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-53608ad7-7af0-4e12-b548-51d7db63c85e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:56.691645 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53608ad7-7af0-4e12-b548-51d7db63c85e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:56.700605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a20fb2-8fda-4a2f-997c-084750060df5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:56.701255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a20fb2-8fda-4a2f-997c-084750060df5 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:56.702157 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a20fb2-8fda-4a2f-997c-084750060df5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:56.704103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a20fb2-8fda-4a2f-997c-084750060df5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:56.731983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a20fb2-8fda-4a2f-997c-084750060df5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:56.735035 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a20fb2-8fda-4a2f-997c-084750060df5 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:56.753163 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1354/2738] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:47:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:47:57.288363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18563377-1f05-4f69-86bd-a4e405c1c11c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:57.291515 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-18563377-1f05-4f69-86bd-a4e405c1c11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:57.360652 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18563377-1f05-4f69-86bd-a4e405c1c11c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:57.370589 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e430f-fb54-4e60-abbf-e958364e8c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:57.371662 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e430f-fb54-4e60-abbf-e958364e8c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:57.372485 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e430f-fb54-4e60-abbf-e958364e8c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:57.374237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e430f-fb54-4e60-abbf-e958364e8c67 admin admin] PATH_INFO: `/v3/projects/ddb11566bf7b45568f0da1d0be7f9132` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:57.376742 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e430f-fb54-4e60-abbf-e958364e8c67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddb11566bf7b45568f0da1d0be7f9132)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:57.384238 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e430f-fb54-4e60-abbf-e958364e8c67 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:57.480813 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ea0e430f-fb54-4e60-abbf-e958364e8c67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddb11566bf7b45568f0da1d0be7f9132', 'request_id': 'req-ea0e430f-fb54-4e60-abbf-e958364e8c67'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:57.523669 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ea0e430f-fb54-4e60-abbf-e958364e8c67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddb11566bf7b45568f0da1d0be7f9132', 'request_id': 'req-ea0e430f-fb54-4e60-abbf-e958364e8c67'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:57.552351 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ea0e430f-fb54-4e60-abbf-e958364e8c67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddb11566bf7b45568f0da1d0be7f9132', 'request_id': 'req-ea0e430f-fb54-4e60-abbf-e958364e8c67'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:57.602526 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ea0e430f-fb54-4e60-abbf-e958364e8c67 admin admin] The token cache is being invalidate because project ddb11566bf7b45568f0da1d0be7f9132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:47:57.609787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ea0e430f-fb54-4e60-abbf-e958364e8c67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:57.621003 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ea0e430f-fb54-4e60-abbf-e958364e8c67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:57.624519 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1385/2739] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:47:57 2022] DELETE /identity/v3/projects/ddb11566bf7b45568f0da1d0be7f9132 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:57.633973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a20fb2-8fda-4a2f-997c-084750060df5 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:57.646859 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a20fb2-8fda-4a2f-997c-084750060df5 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:57.750464 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d125f80-9d27-46cf-b99f-da4c05a7e282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:57.751457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d125f80-9d27-46cf-b99f-da4c05a7e282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:57.753451 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d125f80-9d27-46cf-b99f-da4c05a7e282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:57.753806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d125f80-9d27-46cf-b99f-da4c05a7e282 admin admin] PATH_INFO: `/v3/users/2715f540bfec4ed680c91d4484755409` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:57.755896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d125f80-9d27-46cf-b99f-da4c05a7e282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2715f540bfec4ed680c91d4484755409)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:57.759656 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d125f80-9d27-46cf-b99f-da4c05a7e282 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:57.844505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:57.892219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1d125f80-9d27-46cf-b99f-da4c05a7e282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2715f540bfec4ed680c91d4484755409'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:57.909732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1d125f80-9d27-46cf-b99f-da4c05a7e282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2715f540bfec4ed680c91d4484755409'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:57.927299 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1d125f80-9d27-46cf-b99f-da4c05a7e282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2715f540bfec4ed680c91d4484755409'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:57.960150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1d125f80-9d27-46cf-b99f-da4c05a7e282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2715f540bfec4ed680c91d4484755409'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:57.961539 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1d125f80-9d27-46cf-b99f-da4c05a7e282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2715f540bfec4ed680c91d4484755409'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:57.974286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1355/2740] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:47:57 2022] DELETE /identity/v3/users/2715f540bfec4ed680c91d4484755409 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:58.478941 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e430f-fb54-4e60-abbf-e958364e8c67 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:58.480777 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ea0e430f-fb54-4e60-abbf-e958364e8c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:58.492738 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:58.505138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e430f-fb54-4e60-abbf-e958364e8c67 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:58.610929 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450aa33-1d7f-4b90-9f15-80c5e0318cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:58.611942 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450aa33-1d7f-4b90-9f15-80c5e0318cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:58.612470 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450aa33-1d7f-4b90-9f15-80c5e0318cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:58.612848 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450aa33-1d7f-4b90-9f15-80c5e0318cea admin admin] PATH_INFO: `/v3/projects/8007456682094d64aa3082018a763ef6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:58.614251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450aa33-1d7f-4b90-9f15-80c5e0318cea admin admin] RBAC: Authorizing `identity:delete_project(project_id=8007456682094d64aa3082018a763ef6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:58.616581 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450aa33-1d7f-4b90-9f15-80c5e0318cea admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:58.739274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3450aa33-1d7f-4b90-9f15-80c5e0318cea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8007456682094d64aa3082018a763ef6', 'request_id': 'req-3450aa33-1d7f-4b90-9f15-80c5e0318cea'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:58.749496 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3450aa33-1d7f-4b90-9f15-80c5e0318cea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8007456682094d64aa3082018a763ef6', 'request_id': 'req-3450aa33-1d7f-4b90-9f15-80c5e0318cea'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:58.771156 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3450aa33-1d7f-4b90-9f15-80c5e0318cea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8007456682094d64aa3082018a763ef6', 'request_id': 'req-3450aa33-1d7f-4b90-9f15-80c5e0318cea'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:58.790453 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3450aa33-1d7f-4b90-9f15-80c5e0318cea admin admin] The token cache is being invalidate because project 8007456682094d64aa3082018a763ef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:47:58.790991 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3450aa33-1d7f-4b90-9f15-80c5e0318cea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:58.792348 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3450aa33-1d7f-4b90-9f15-80c5e0318cea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:58.814103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1386/2741] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:47:58 2022] DELETE /identity/v3/projects/8007456682094d64aa3082018a763ef6 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:47:58.815735 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d125f80-9d27-46cf-b99f-da4c05a7e282 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:58.817337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1d125f80-9d27-46cf-b99f-da4c05a7e282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:58.842652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d125f80-9d27-46cf-b99f-da4c05a7e282 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:58.963363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286c5f0-0794-46d9-89d9-541fc223bc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:58.964707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286c5f0-0794-46d9-89d9-541fc223bc4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:58.965203 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286c5f0-0794-46d9-89d9-541fc223bc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:58.965781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286c5f0-0794-46d9-89d9-541fc223bc4d admin admin] PATH_INFO: `/v3/users/ef4b6f15ebc24040bda557b6340ef543` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:58.968168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1286c5f0-0794-46d9-89d9-541fc223bc4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef4b6f15ebc24040bda557b6340ef543)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:58.970498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1286c5f0-0794-46d9-89d9-541fc223bc4d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:59.117998 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1286c5f0-0794-46d9-89d9-541fc223bc4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef4b6f15ebc24040bda557b6340ef543'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:59.134749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1286c5f0-0794-46d9-89d9-541fc223bc4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef4b6f15ebc24040bda557b6340ef543'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:59.146015 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1286c5f0-0794-46d9-89d9-541fc223bc4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef4b6f15ebc24040bda557b6340ef543'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:59.198313 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1286c5f0-0794-46d9-89d9-541fc223bc4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef4b6f15ebc24040bda557b6340ef543'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:59.199055 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1286c5f0-0794-46d9-89d9-541fc223bc4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef4b6f15ebc24040bda557b6340ef543'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:59.211085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1356/2742] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:47:58 2022] DELETE /identity/v3/users/ef4b6f15ebc24040bda557b6340ef543 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:47:59.587894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450aa33-1d7f-4b90-9f15-80c5e0318cea admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:47:59.590089 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3450aa33-1d7f-4b90-9f15-80c5e0318cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:47:59.607421 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450aa33-1d7f-4b90-9f15-80c5e0318cea admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:47:59.710885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2901f1e-4845-49ca-8656-02d35bdcc545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:47:59.710885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2901f1e-4845-49ca-8656-02d35bdcc545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:47:59.710885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2901f1e-4845-49ca-8656-02d35bdcc545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:47:59.710885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2901f1e-4845-49ca-8656-02d35bdcc545 admin admin] PATH_INFO: `/v3/projects/395606b2e96c4b3d9b6c833dd3b8eb3b` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:47:59.711349 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2901f1e-4845-49ca-8656-02d35bdcc545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=395606b2e96c4b3d9b6c833dd3b8eb3b)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:47:59.711349 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2901f1e-4845-49ca-8656-02d35bdcc545 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:47:59.771489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:47:59.786310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f2901f1e-4845-49ca-8656-02d35bdcc545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '395606b2e96c4b3d9b6c833dd3b8eb3b', 'request_id': 'req-f2901f1e-4845-49ca-8656-02d35bdcc545'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:59.799859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f2901f1e-4845-49ca-8656-02d35bdcc545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '395606b2e96c4b3d9b6c833dd3b8eb3b', 'request_id': 'req-f2901f1e-4845-49ca-8656-02d35bdcc545'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:59.828592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f2901f1e-4845-49ca-8656-02d35bdcc545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '395606b2e96c4b3d9b6c833dd3b8eb3b', 'request_id': 'req-f2901f1e-4845-49ca-8656-02d35bdcc545'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:59.857140 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f2901f1e-4845-49ca-8656-02d35bdcc545 admin admin] The token cache is being invalidate because project 395606b2e96c4b3d9b6c833dd3b8eb3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:47:59.858474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f2901f1e-4845-49ca-8656-02d35bdcc545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:59.859989 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f2901f1e-4845-49ca-8656-02d35bdcc545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:47:59.863664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1387/2743] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:47:59 2022] DELETE /identity/v3/projects/395606b2e96c4b3d9b6c833dd3b8eb3b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:48:04.889431 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286c5f0-0794-46d9-89d9-541fc223bc4d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:48:04.891030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1286c5f0-0794-46d9-89d9-541fc223bc4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:48:04.931673 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:48:05.012183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286c5f0-0794-46d9-89d9-541fc223bc4d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:48:05.094889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e2cad9-4473-43f8-9d7c-7f5de2b7abc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:48:05.095859 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2cad9-4473-43f8-9d7c-7f5de2b7abc0 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:48:05.096454 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2cad9-4473-43f8-9d7c-7f5de2b7abc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:48:05.096819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e2cad9-4473-43f8-9d7c-7f5de2b7abc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:48:05.101446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-32e2cad9-4473-43f8-9d7c-7f5de2b7abc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:48:05.193112 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e2cad9-4473-43f8-9d7c-7f5de2b7abc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:48:05.195258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e2cad9-4473-43f8-9d7c-7f5de2b7abc0 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:48:05.219852 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1357/2744] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:48:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:48:05.555322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2adb70-e5c5-43f0-ac7d-4faf39aa3a4a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:48:05.555703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2adb70-e5c5-43f0-ac7d-4faf39aa3a4a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:48:05.555950 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2adb70-e5c5-43f0-ac7d-4faf39aa3a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:48:05.577738 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-cc2adb70-e5c5-43f0-ac7d-4faf39aa3a4a None None] MFA Rules not processed for user `70eea309b21d4e628d14991a68ceeeef`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:48:05.579011 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cc2adb70-e5c5-43f0-ac7d-4faf39aa3a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:48:05.652469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1388/2745] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:48:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:48:12.977310 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdb7f6-843d-4612-9942-796373051f66 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:48:12.978841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdb7f6-843d-4612-9942-796373051f66 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:48:12.979430 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdb7f6-843d-4612-9942-796373051f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:48:13.003005 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-77bdb7f6-843d-4612-9942-796373051f66 None None] MFA Rules not processed for user `257450db238d4efda783e30cbcd23d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:48:13.004300 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-77bdb7f6-843d-4612-9942-796373051f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:48:13.057147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:48:13.067569 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1358/2746] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jul 21 09:48:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:48:13.081114 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2adb70-e5c5-43f0-ac7d-4faf39aa3a4a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:48:13.082604 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cc2adb70-e5c5-43f0-ac7d-4faf39aa3a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:48:13.155850 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2adb70-e5c5-43f0-ac7d-4faf39aa3a4a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:48:13.161403 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9e1c7-6407-4985-9328-ec36936872f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061def74ec7f42ce9a174e0944bf499f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:48:13.161600 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9e1c7-6407-4985-9328-ec36936872f5 service swift] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:48:13.161884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9e1c7-6407-4985-9328-ec36936872f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:48:13.162167 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9e1c7-6407-4985-9328-ec36936872f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:48:13.172826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9e1c7-6407-4985-9328-ec36936872f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:48:13.174459 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9e1c7-6407-4985-9328-ec36936872f5 service swift] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:48:13.183482 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1389/2747] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jul 21 09:48:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 09:48:23.061949 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:48:23.179787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=5,keystone:SELECT=68,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:48:35.197658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdb7f6-843d-4612-9942-796373051f66 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:48:35.199787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-77bdb7f6-843d-4612-9942-796373051f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:48:35.265609 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdb7f6-843d-4612-9942-796373051f66 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:48:35.270491 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58702a-a737-431b-9c01-9ea93b21c66f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:48:35.271133 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58702a-a737-431b-9c01-9ea93b21c66f service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:48:35.271413 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58702a-a737-431b-9c01-9ea93b21c66f service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:48:35.271686 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad58702a-a737-431b-9c01-9ea93b21c66f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:48:35.273489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ad58702a-a737-431b-9c01-9ea93b21c66f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:48:35.284351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58702a-a737-431b-9c01-9ea93b21c66f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:48:35.285794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad58702a-a737-431b-9c01-9ea93b21c66f service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:48:35.294624 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1359/2748] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:48:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:48:45.290372 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=26 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:48:57.349670 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9e1c7-6407-4985-9328-ec36936872f5 service swift] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:48:57.359022 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9e1c7-6407-4985-9328-ec36936872f5 service swift] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:48:57.363248 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec20962-638c-4bcf-ab8d-fdac07478f1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:48:57.363881 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec20962-638c-4bcf-ab8d-fdac07478f1d service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:48:57.364189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec20962-638c-4bcf-ab8d-fdac07478f1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:48:57.364466 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec20962-638c-4bcf-ab8d-fdac07478f1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:48:57.366189 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1ec20962-638c-4bcf-ab8d-fdac07478f1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:48:57.431066 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec20962-638c-4bcf-ab8d-fdac07478f1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:48:57.432562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec20962-638c-4bcf-ab8d-fdac07478f1d service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:48:57.444713 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1390/2749] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:48:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:48:58.539875 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58702a-a737-431b-9c01-9ea93b21c66f service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:48:58.542216 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ad58702a-a737-431b-9c01-9ea93b21c66f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:48:58.554908 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad58702a-a737-431b-9c01-9ea93b21c66f service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:48:58.558772 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf356fb-c898-41d0-81d4-a74706ea7092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:48:58.559512 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf356fb-c898-41d0-81d4-a74706ea7092 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:48:58.560060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf356fb-c898-41d0-81d4-a74706ea7092 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:48:58.560442 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf356fb-c898-41d0-81d4-a74706ea7092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:48:58.562520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-faf356fb-c898-41d0-81d4-a74706ea7092 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:48:58.642886 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf356fb-c898-41d0-81d4-a74706ea7092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:48:58.644414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf356fb-c898-41d0-81d4-a74706ea7092 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:48:58.660133 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1360/2750] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:48:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:49:07.437826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=30 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:49:08.651191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=30 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:49:13.309131 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec20962-638c-4bcf-ab8d-fdac07478f1d service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:49:13.310563 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1ec20962-638c-4bcf-ab8d-fdac07478f1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:49:13.419865 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec20962-638c-4bcf-ab8d-fdac07478f1d service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:49:13.425480 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200e2d2-b5ff-40d9-b582-eb3cbba4fdd0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:49:13.426116 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200e2d2-b5ff-40d9-b582-eb3cbba4fdd0 service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:49:13.426402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200e2d2-b5ff-40d9-b582-eb3cbba4fdd0 service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:49:13.426674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200e2d2-b5ff-40d9-b582-eb3cbba4fdd0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:49:13.428432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8200e2d2-b5ff-40d9-b582-eb3cbba4fdd0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:49:13.440205 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200e2d2-b5ff-40d9-b582-eb3cbba4fdd0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:49:13.441790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200e2d2-b5ff-40d9-b582-eb3cbba4fdd0 service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:49:13.452008 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1391/2751] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:49:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:49:23.447368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=48 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:51:30.135820 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf356fb-c898-41d0-81d4-a74706ea7092 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:51:30.137373 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-faf356fb-c898-41d0-81d4-a74706ea7092 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:30.284033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf356fb-c898-41d0-81d4-a74706ea7092 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:51:30.291593 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e71ba1-e48c-40ed-8cab-ce062f29144e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:51:30.292254 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e71ba1-e48c-40ed-8cab-ce062f29144e service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:30.292545 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e71ba1-e48c-40ed-8cab-ce062f29144e service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:30.292825 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e71ba1-e48c-40ed-8cab-ce062f29144e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:30.295136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a6e71ba1-e48c-40ed-8cab-ce062f29144e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:30.314132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e71ba1-e48c-40ed-8cab-ce062f29144e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:51:30.316966 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e71ba1-e48c-40ed-8cab-ce062f29144e service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:51:30.332287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1361/2752] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:51:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:51:38.215227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200e2d2-b5ff-40d9-b582-eb3cbba4fdd0 service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:51:38.216669 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8200e2d2-b5ff-40d9-b582-eb3cbba4fdd0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:38.268854 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200e2d2-b5ff-40d9-b582-eb3cbba4fdd0 service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:51:38.274450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f009b-87b6-424c-9b3c-8ff367a3ec0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:51:38.275099 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f009b-87b6-424c-9b3c-8ff367a3ec0a service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:38.275386 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f009b-87b6-424c-9b3c-8ff367a3ec0a service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:38.275674 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f009b-87b6-424c-9b3c-8ff367a3ec0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:38.277487 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fb1f009b-87b6-424c-9b3c-8ff367a3ec0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:38.291036 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f009b-87b6-424c-9b3c-8ff367a3ec0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:51:38.292829 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f009b-87b6-424c-9b3c-8ff367a3ec0a service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:51:38.304953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1392/2753] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:51:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:51:40.270509 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f8860-44f1-496c-a9e0-d259a2f32925 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:40.271446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f8860-44f1-496c-a9e0-d259a2f32925 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:40.271446 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f8860-44f1-496c-a9e0-d259a2f32925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:40.290125 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-be8f8860-44f1-496c-a9e0-d259a2f32925 None None] MFA Rules not processed for user `257450db238d4efda783e30cbcd23d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:51:40.291321 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-be8f8860-44f1-496c-a9e0-d259a2f32925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:40.298762 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1362/2754] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jul 21 09:51:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:51:40.339428 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f009b-87b6-424c-9b3c-8ff367a3ec0a service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:51:40.352104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f009b-87b6-424c-9b3c-8ff367a3ec0a service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:51:40.357363 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffaa41-a574-49c5-8fd7-271e4b40c97e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '061def74ec7f42ce9a174e0944bf499f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:51:40.358029 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffaa41-a574-49c5-8fd7-271e4b40c97e service swift] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:40.358316 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffaa41-a574-49c5-8fd7-271e4b40c97e service swift] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:40.358594 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffaa41-a574-49c5-8fd7-271e4b40c97e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:40.368031 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffaa41-a574-49c5-8fd7-271e4b40c97e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:51:40.369676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffaa41-a574-49c5-8fd7-271e4b40c97e service swift] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:51:40.378560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1393/2755] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jul 21 09:51:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 09:51:41.127053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f8860-44f1-496c-a9e0-d259a2f32925 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:51:41.147193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f8860-44f1-496c-a9e0-d259a2f32925 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:51:41.150851 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb45ac-cb05-478b-8121-f3474d7aab1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:51:41.151496 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb45ac-cb05-478b-8121-f3474d7aab1f service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:41.151781 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb45ac-cb05-478b-8121-f3474d7aab1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:41.152096 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb45ac-cb05-478b-8121-f3474d7aab1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:41.153883 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9abb45ac-cb05-478b-8121-f3474d7aab1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:41.164194 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb45ac-cb05-478b-8121-f3474d7aab1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:51:41.165670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb45ac-cb05-478b-8121-f3474d7aab1f service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:51:41.174863 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1363/2756] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:51:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:51:41.921969 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffaa41-a574-49c5-8fd7-271e4b40c97e service swift] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:51:41.936247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffaa41-a574-49c5-8fd7-271e4b40c97e service swift] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:51:41.943227 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c266af-84b6-43f7-9642-57bd728d4bd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:51:41.944222 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c266af-84b6-43f7-9642-57bd728d4bd2 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:41.944504 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c266af-84b6-43f7-9642-57bd728d4bd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:41.944783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c266af-84b6-43f7-9642-57bd728d4bd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:41.946707 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-45c266af-84b6-43f7-9642-57bd728d4bd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:41.960391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c266af-84b6-43f7-9642-57bd728d4bd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:51:41.962310 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c266af-84b6-43f7-9642-57bd728d4bd2 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:51:41.975784 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1394/2757] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:51:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:51:42.299396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d4bbc-2194-46de-a8d1-42ce5cd0461b None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:42.300019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d4bbc-2194-46de-a8d1-42ce5cd0461b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:42.300019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d4bbc-2194-46de-a8d1-42ce5cd0461b None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:42.302538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1364/2758] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jul 21 09:51:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:51:42.309295 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167228f-9b02-4142-91a8-375f5adaf637 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:42.309883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167228f-9b02-4142-91a8-375f5adaf637 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:42.310169 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167228f-9b02-4142-91a8-375f5adaf637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:42.353907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a167228f-9b02-4142-91a8-375f5adaf637 None None] MFA Rules not processed for user `9e2a13c2cb864175a082de3ca8b166c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:51:42.355138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a167228f-9b02-4142-91a8-375f5adaf637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:42.362257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1395/2759] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jul 21 09:51:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:51:42.381664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d4bbc-2194-46de-a8d1-42ce5cd0461b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:51:42.395647 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d4bbc-2194-46de-a8d1-42ce5cd0461b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:51:42.400784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87ec4a-6894-45f1-93f6-9a7a8ac1a5ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:51:42.401432 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87ec4a-6894-45f1-93f6-9a7a8ac1a5ea service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:42.401738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87ec4a-6894-45f1-93f6-9a7a8ac1a5ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:42.402161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87ec4a-6894-45f1-93f6-9a7a8ac1a5ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:42.410261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87ec4a-6894-45f1-93f6-9a7a8ac1a5ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:51:42.413001 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87ec4a-6894-45f1-93f6-9a7a8ac1a5ea service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:51:42.426769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1365/2760] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:51:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:51:42.528298 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49c803-e0c3-42a5-90b7-1dd0abc1b75f None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:42.528658 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49c803-e0c3-42a5-90b7-1dd0abc1b75f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:42.529011 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49c803-e0c3-42a5-90b7-1dd0abc1b75f None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:42.530440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1396/2761] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jul 21 09:51:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 09:51:42.542246 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bed33-4a4d-4f77-ac10-bdbed503dd8b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:42.542546 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bed33-4a4d-4f77-ac10-bdbed503dd8b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:42.543012 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057bed33-4a4d-4f77-ac10-bdbed503dd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:42.589096 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-057bed33-4a4d-4f77-ac10-bdbed503dd8b None None] MFA Rules not processed for user `c48d41dcf9b344a8b2fec77f4dd75fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:51:42.590410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-057bed33-4a4d-4f77-ac10-bdbed503dd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:42.601271 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1366/2762] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jul 21 09:51:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:51:42.616751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49c803-e0c3-42a5-90b7-1dd0abc1b75f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:51:42.628570 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49c803-e0c3-42a5-90b7-1dd0abc1b75f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:51:42.638586 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77004c-bd94-4354-940a-e2458acec558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:51:42.639535 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77004c-bd94-4354-940a-e2458acec558 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:42.640187 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77004c-bd94-4354-940a-e2458acec558 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:42.640565 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77004c-bd94-4354-940a-e2458acec558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:42.654749 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77004c-bd94-4354-940a-e2458acec558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:51:42.656337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77004c-bd94-4354-940a-e2458acec558 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:51:42.668519 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1397/2763] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:51:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:51:42.749794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057bed33-4a4d-4f77-ac10-bdbed503dd8b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:51:42.752691 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-057bed33-4a4d-4f77-ac10-bdbed503dd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:42.778802 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057bed33-4a4d-4f77-ac10-bdbed503dd8b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:51:42.787204 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5f9f0-5ae6-4147-be48-94ed664d5c95 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:51:42.789317 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5f9f0-5ae6-4147-be48-94ed664d5c95 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:42.789925 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5f9f0-5ae6-4147-be48-94ed664d5c95 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:42.790501 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5f9f0-5ae6-4147-be48-94ed664d5c95 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:42.793794 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f5d5f9f0-5ae6-4147-be48-94ed664d5c95 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:42.810300 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5f9f0-5ae6-4147-be48-94ed664d5c95 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:51:42.811997 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5f9f0-5ae6-4147-be48-94ed664d5c95 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:51:42.826147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1367/2764] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:51:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:51:47.701603 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77004c-bd94-4354-940a-e2458acec558 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:51:47.721058 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77004c-bd94-4354-940a-e2458acec558 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:51:47.727115 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5309a3d-55e5-4384-a948-95283a90ee1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:51:47.727772 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5309a3d-55e5-4384-a948-95283a90ee1a service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:47.728092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5309a3d-55e5-4384-a948-95283a90ee1a service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:47.728373 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5309a3d-55e5-4384-a948-95283a90ee1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:47.730173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d5309a3d-55e5-4384-a948-95283a90ee1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:47.745319 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5309a3d-55e5-4384-a948-95283a90ee1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:51:47.745319 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5309a3d-55e5-4384-a948-95283a90ee1a service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:51:47.760438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1398/2765] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:51:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:51:47.765837 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5f9f0-5ae6-4147-be48-94ed664d5c95 service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:51:47.767899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f5d5f9f0-5ae6-4147-be48-94ed664d5c95 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:47.863152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:51:47.879931 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5f9f0-5ae6-4147-be48-94ed664d5c95 service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:51:47.884328 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac628a-4727-467f-94a4-69764f2eea06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:51:47.885237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac628a-4727-467f-94a4-69764f2eea06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:47.885544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac628a-4727-467f-94a4-69764f2eea06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:47.885795 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac628a-4727-467f-94a4-69764f2eea06 admin admin] PATH_INFO: `/v3/users/70eea309b21d4e628d14991a68ceeeef` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:47.887044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac628a-4727-467f-94a4-69764f2eea06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70eea309b21d4e628d14991a68ceeeef)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:51:47.889236 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac628a-4727-467f-94a4-69764f2eea06 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:51:47.945986 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b9ac628a-4727-467f-94a4-69764f2eea06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70eea309b21d4e628d14991a68ceeeef'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:47.953552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b9ac628a-4727-467f-94a4-69764f2eea06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70eea309b21d4e628d14991a68ceeeef'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:47.962528 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b9ac628a-4727-467f-94a4-69764f2eea06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70eea309b21d4e628d14991a68ceeeef'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:47.975732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b9ac628a-4727-467f-94a4-69764f2eea06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70eea309b21d4e628d14991a68ceeeef'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:47.976645 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b9ac628a-4727-467f-94a4-69764f2eea06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70eea309b21d4e628d14991a68ceeeef'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:47.983292 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1368/2766] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:51:47 2022] DELETE /identity/v3/users/70eea309b21d4e628d14991a68ceeeef => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:51:48.169536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5309a3d-55e5-4384-a948-95283a90ee1a service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:51:48.181274 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5309a3d-55e5-4384-a948-95283a90ee1a service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:51:48.241522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7caa4-0cb5-4811-893a-416842fb71f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:51:48.242383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7caa4-0cb5-4811-893a-416842fb71f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:48.242687 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7caa4-0cb5-4811-893a-416842fb71f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:48.242946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7caa4-0cb5-4811-893a-416842fb71f6 admin admin] PATH_INFO: `/v3/projects/253b656640304948a74e7bcc33b54561` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:48.244159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7caa4-0cb5-4811-893a-416842fb71f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=253b656640304948a74e7bcc33b54561)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:51:48.246087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7caa4-0cb5-4811-893a-416842fb71f6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:51:48.300095 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:51:48.305178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b9f7caa4-0cb5-4811-893a-416842fb71f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '253b656640304948a74e7bcc33b54561', 'request_id': 'req-b9f7caa4-0cb5-4811-893a-416842fb71f6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:48.311135 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b9f7caa4-0cb5-4811-893a-416842fb71f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '253b656640304948a74e7bcc33b54561', 'request_id': 'req-b9f7caa4-0cb5-4811-893a-416842fb71f6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:48.317998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b9f7caa4-0cb5-4811-893a-416842fb71f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '253b656640304948a74e7bcc33b54561', 'request_id': 'req-b9f7caa4-0cb5-4811-893a-416842fb71f6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:48.330863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b9f7caa4-0cb5-4811-893a-416842fb71f6 admin admin] The token cache is being invalidate because project 253b656640304948a74e7bcc33b54561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:51:48.331182 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b9f7caa4-0cb5-4811-893a-416842fb71f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:48.331828 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b9f7caa4-0cb5-4811-893a-416842fb71f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:48.333729 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1399/2767] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:51:48 2022] DELETE /identity/v3/projects/253b656640304948a74e7bcc33b54561 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:51:48.337822 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac628a-4727-467f-94a4-69764f2eea06 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:51:48.339207 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b9ac628a-4727-467f-94a4-69764f2eea06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:48.352191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac628a-4727-467f-94a4-69764f2eea06 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:51:48.408697 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195ea9a-fcaf-43d7-8dd3-4b56d623b153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:51:48.409570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195ea9a-fcaf-43d7-8dd3-4b56d623b153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:48.409855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195ea9a-fcaf-43d7-8dd3-4b56d623b153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:48.410173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195ea9a-fcaf-43d7-8dd3-4b56d623b153 admin admin] PATH_INFO: `/v3/users/9cce43e8533e4a6e96800538aa90d8d1` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:48.411396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195ea9a-fcaf-43d7-8dd3-4b56d623b153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cce43e8533e4a6e96800538aa90d8d1)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:51:48.413385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195ea9a-fcaf-43d7-8dd3-4b56d623b153 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:51:48.459368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d195ea9a-fcaf-43d7-8dd3-4b56d623b153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cce43e8533e4a6e96800538aa90d8d1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:48.467101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d195ea9a-fcaf-43d7-8dd3-4b56d623b153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cce43e8533e4a6e96800538aa90d8d1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:48.475015 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d195ea9a-fcaf-43d7-8dd3-4b56d623b153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cce43e8533e4a6e96800538aa90d8d1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:48.480237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:51:48.489441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d195ea9a-fcaf-43d7-8dd3-4b56d623b153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cce43e8533e4a6e96800538aa90d8d1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:48.490109 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d195ea9a-fcaf-43d7-8dd3-4b56d623b153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cce43e8533e4a6e96800538aa90d8d1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:48.496005 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1369/2768] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:51:48 2022] DELETE /identity/v3/users/9cce43e8533e4a6e96800538aa90d8d1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:51:48.690732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7caa4-0cb5-4811-893a-416842fb71f6 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:51:48.693531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b9f7caa4-0cb5-4811-893a-416842fb71f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:48.717693 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7caa4-0cb5-4811-893a-416842fb71f6 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:51:48.782009 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:51:48.782871 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:48.783158 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:48.783438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7 admin admin] PATH_INFO: `/v3/projects/82561dda7cff4e9eb8cfa6b0a46ed989` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:48.784582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82561dda7cff4e9eb8cfa6b0a46ed989)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:51:48.786540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:51:48.834578 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82561dda7cff4e9eb8cfa6b0a46ed989', 'request_id': 'req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:48.840270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82561dda7cff4e9eb8cfa6b0a46ed989', 'request_id': 'req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:48.846532 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82561dda7cff4e9eb8cfa6b0a46ed989', 'request_id': 'req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:48.860661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7 admin admin] The token cache is being invalidate because project 82561dda7cff4e9eb8cfa6b0a46ed989 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:51:48.861086 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:48.861793 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:51:48.863425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1400/2769] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:51:48 2022] DELETE /identity/v3/projects/82561dda7cff4e9eb8cfa6b0a46ed989 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:51:53.247171 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195ea9a-fcaf-43d7-8dd3-4b56d623b153 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:51:53.248640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d195ea9a-fcaf-43d7-8dd3-4b56d623b153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:53.306759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195ea9a-fcaf-43d7-8dd3-4b56d623b153 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:51:53.361458 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a6cee-8d99-4329-ac11-eac094b7ab23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:51:53.362099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a6cee-8d99-4329-ac11-eac094b7ab23 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:53.362380 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a6cee-8d99-4329-ac11-eac094b7ab23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:53.362654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a6cee-8d99-4329-ac11-eac094b7ab23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:53.364543 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b53a6cee-8d99-4329-ac11-eac094b7ab23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:53.434505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a6cee-8d99-4329-ac11-eac094b7ab23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:51:53.436002 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a6cee-8d99-4329-ac11-eac094b7ab23 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:51:53.448356 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1370/2770] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:51:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:51:54.625242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:51:54.626664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:54.640329 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de7296-6ab2-4084-9ac1-97fbe0ba18f7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:51:54.643830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23f4d6-3b64-49b9-a5a8-fa02eb40a874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:51:54.645101 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23f4d6-3b64-49b9-a5a8-fa02eb40a874 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:51:54.645383 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23f4d6-3b64-49b9-a5a8-fa02eb40a874 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:51:54.645661 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23f4d6-3b64-49b9-a5a8-fa02eb40a874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:51:54.647438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ae23f4d6-3b64-49b9-a5a8-fa02eb40a874 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:51:54.661106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23f4d6-3b64-49b9-a5a8-fa02eb40a874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:51:54.662533 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23f4d6-3b64-49b9-a5a8-fa02eb40a874 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:51:54.673832 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1401/2771] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:51:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:52:03.442802 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=79,keystone:DELETE=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:52:04.668856 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=65,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:53:05.974397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a6cee-8d99-4329-ac11-eac094b7ab23 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:53:05.975894 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b53a6cee-8d99-4329-ac11-eac094b7ab23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:53:06.045023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a6cee-8d99-4329-ac11-eac094b7ab23 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:53:06.048652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c02b8d-2c58-419d-bdd7-280ecc2efb14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:53:06.049286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c02b8d-2c58-419d-bdd7-280ecc2efb14 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:53:06.049568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c02b8d-2c58-419d-bdd7-280ecc2efb14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:53:06.049840 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c02b8d-2c58-419d-bdd7-280ecc2efb14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:53:06.051580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-91c02b8d-2c58-419d-bdd7-280ecc2efb14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:53:06.112861 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c02b8d-2c58-419d-bdd7-280ecc2efb14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:53:06.114369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c02b8d-2c58-419d-bdd7-280ecc2efb14 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:53:06.124327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1371/2772] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:53:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:53:16.119906 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=52 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:53:33.352953 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23f4d6-3b64-49b9-a5a8-fa02eb40a874 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:53:33.361679 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23f4d6-3b64-49b9-a5a8-fa02eb40a874 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:53:33.365243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a631e-8666-42ae-bb4e-400326618111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:53:33.365876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a631e-8666-42ae-bb4e-400326618111 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:53:33.366157 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a631e-8666-42ae-bb4e-400326618111 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:53:33.366430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a631e-8666-42ae-bb4e-400326618111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:53:33.368443 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ed4a631e-8666-42ae-bb4e-400326618111 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:53:33.428154 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a631e-8666-42ae-bb4e-400326618111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:53:33.429611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a631e-8666-42ae-bb4e-400326618111 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:53:33.439199 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1402/2773] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:53:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:53:35.494738 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c02b8d-2c58-419d-bdd7-280ecc2efb14 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:53:35.496275 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-91c02b8d-2c58-419d-bdd7-280ecc2efb14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:53:35.508014 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c02b8d-2c58-419d-bdd7-280ecc2efb14 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:53:35.512364 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0178bde-9e5b-4d7c-be57-eb9068efebbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:53:35.513003 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0178bde-9e5b-4d7c-be57-eb9068efebbf service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:53:35.513292 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0178bde-9e5b-4d7c-be57-eb9068efebbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:53:35.513573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0178bde-9e5b-4d7c-be57-eb9068efebbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:53:35.515314 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e0178bde-9e5b-4d7c-be57-eb9068efebbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:53:35.524693 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0178bde-9e5b-4d7c-be57-eb9068efebbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:53:35.526133 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0178bde-9e5b-4d7c-be57-eb9068efebbf service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:53:35.534169 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1372/2774] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:53:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:53:38.144947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a631e-8666-42ae-bb4e-400326618111 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:53:38.155555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a631e-8666-42ae-bb4e-400326618111 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:53:38.160905 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060379d-4080-46fa-a2af-4dd97b7d3075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:53:38.161556 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060379d-4080-46fa-a2af-4dd97b7d3075 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:53:38.162228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060379d-4080-46fa-a2af-4dd97b7d3075 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:53:38.162947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060379d-4080-46fa-a2af-4dd97b7d3075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:53:38.164799 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2060379d-4080-46fa-a2af-4dd97b7d3075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:53:38.230598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060379d-4080-46fa-a2af-4dd97b7d3075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:53:38.232069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060379d-4080-46fa-a2af-4dd97b7d3075 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:53:38.241593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1403/2775] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:53:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:53:44.598615 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0178bde-9e5b-4d7c-be57-eb9068efebbf service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:53:44.600160 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e0178bde-9e5b-4d7c-be57-eb9068efebbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:53:44.661815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060379d-4080-46fa-a2af-4dd97b7d3075 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:53:44.663906 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2060379d-4080-46fa-a2af-4dd97b7d3075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:53:44.685061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060379d-4080-46fa-a2af-4dd97b7d3075 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:53:44.692398 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0178bde-9e5b-4d7c-be57-eb9068efebbf service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:53:44.694210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098d27a-fe5e-428d-a42b-b407a363e468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:53:44.695150 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098d27a-fe5e-428d-a42b-b407a363e468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:53:44.695444 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098d27a-fe5e-428d-a42b-b407a363e468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:53:44.695838 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098d27a-fe5e-428d-a42b-b407a363e468 admin admin] PATH_INFO: `/v3/users/9d439ed3cd1f4610a4d4902b42d19f37` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:53:44.697217 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098d27a-fe5e-428d-a42b-b407a363e468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d439ed3cd1f4610a4d4902b42d19f37)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:53:44.698145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98fc60-00a4-46f2-8eda-0ef42845298e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:53:44.698800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98fc60-00a4-46f2-8eda-0ef42845298e service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:53:44.699083 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98fc60-00a4-46f2-8eda-0ef42845298e service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:53:44.699181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098d27a-fe5e-428d-a42b-b407a363e468 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:53:44.699359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98fc60-00a4-46f2-8eda-0ef42845298e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:53:44.701152 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1d98fc60-00a4-46f2-8eda-0ef42845298e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:53:44.718777 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98fc60-00a4-46f2-8eda-0ef42845298e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:53:44.720471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98fc60-00a4-46f2-8eda-0ef42845298e service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:53:44.734585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1373/2776] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:53:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:53:44.763611 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7098d27a-fe5e-428d-a42b-b407a363e468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d439ed3cd1f4610a4d4902b42d19f37'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:44.764326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7098d27a-fe5e-428d-a42b-b407a363e468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d439ed3cd1f4610a4d4902b42d19f37'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:44.773766 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7098d27a-fe5e-428d-a42b-b407a363e468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d439ed3cd1f4610a4d4902b42d19f37'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:44.787339 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7098d27a-fe5e-428d-a42b-b407a363e468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d439ed3cd1f4610a4d4902b42d19f37'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:44.797375 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7098d27a-fe5e-428d-a42b-b407a363e468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d439ed3cd1f4610a4d4902b42d19f37'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:44.804184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1404/2777] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:53:44 2022] DELETE /identity/v3/users/9d439ed3cd1f4610a4d4902b42d19f37 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:53:44.992987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98fc60-00a4-46f2-8eda-0ef42845298e service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:53:45.004747 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98fc60-00a4-46f2-8eda-0ef42845298e service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:53:45.063740 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:53:45.064550 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:53:45.064834 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:53:45.065117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e admin admin] PATH_INFO: `/v3/projects/cff5412db5c24a03a4ca18ec2780d77c` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:53:45.066294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cff5412db5c24a03a4ca18ec2780d77c)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:53:45.068248 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:53:45.115750 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cff5412db5c24a03a4ca18ec2780d77c', 'request_id': 'req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:45.122468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cff5412db5c24a03a4ca18ec2780d77c', 'request_id': 'req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:45.130221 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cff5412db5c24a03a4ca18ec2780d77c', 'request_id': 'req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:45.142176 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e admin admin] The token cache is being invalidate because project cff5412db5c24a03a4ca18ec2780d77c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:53:45.142463 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:45.143095 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:45.144694 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1374/2778] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:53:44 2022] DELETE /identity/v3/projects/cff5412db5c24a03a4ca18ec2780d77c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:53:45.148936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098d27a-fe5e-428d-a42b-b407a363e468 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:53:45.151103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7098d27a-fe5e-428d-a42b-b407a363e468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:53:45.160815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:53:45.164804 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098d27a-fe5e-428d-a42b-b407a363e468 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:53:45.226347 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dcb1c-05b1-422d-8fc5-bf248c0e5554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:53:45.227256 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dcb1c-05b1-422d-8fc5-bf248c0e5554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:53:45.227541 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dcb1c-05b1-422d-8fc5-bf248c0e5554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:53:45.227818 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dcb1c-05b1-422d-8fc5-bf248c0e5554 admin admin] PATH_INFO: `/v3/users/86830104a3524a30aea4d5c5ecc241dc` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:53:45.229644 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dcb1c-05b1-422d-8fc5-bf248c0e5554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86830104a3524a30aea4d5c5ecc241dc)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:53:45.231571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165dcb1c-05b1-422d-8fc5-bf248c0e5554 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:53:45.282372 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-165dcb1c-05b1-422d-8fc5-bf248c0e5554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86830104a3524a30aea4d5c5ecc241dc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:45.283033 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-165dcb1c-05b1-422d-8fc5-bf248c0e5554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86830104a3524a30aea4d5c5ecc241dc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:45.291025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-165dcb1c-05b1-422d-8fc5-bf248c0e5554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86830104a3524a30aea4d5c5ecc241dc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:45.303401 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-165dcb1c-05b1-422d-8fc5-bf248c0e5554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86830104a3524a30aea4d5c5ecc241dc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:45.310695 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-165dcb1c-05b1-422d-8fc5-bf248c0e5554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86830104a3524a30aea4d5c5ecc241dc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:45.316623 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1405/2779] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:53:45 2022] DELETE /identity/v3/users/86830104a3524a30aea4d5c5ecc241dc => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:53:45.491702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:53:45.493180 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:53:45.506708 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49764267-e9b5-4fb1-a3a0-eb01b39f1f6e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:53:45.511568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:53:45.569013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:53:45.569876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:53:45.570161 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:53:45.570434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14 admin admin] PATH_INFO: `/v3/projects/348dca2fbc924780808fa77de996a3fe` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:53:45.571605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=348dca2fbc924780808fa77de996a3fe)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:53:45.574271 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:53:45.622410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '348dca2fbc924780808fa77de996a3fe', 'request_id': 'req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:45.628866 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '348dca2fbc924780808fa77de996a3fe', 'request_id': 'req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:45.637655 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '348dca2fbc924780808fa77de996a3fe', 'request_id': 'req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:45.648744 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14 admin admin] The token cache is being invalidate because project 348dca2fbc924780808fa77de996a3fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:53:45.649030 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:45.649665 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:53:45.651489 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1375/2780] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:53:45 2022] DELETE /identity/v3/projects/348dca2fbc924780808fa77de996a3fe => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:53:55.313104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:53:55.645498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:54:15.277736 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dcb1c-05b1-422d-8fc5-bf248c0e5554 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:54:15.279242 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-165dcb1c-05b1-422d-8fc5-bf248c0e5554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:54:15.348900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dcb1c-05b1-422d-8fc5-bf248c0e5554 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:54:15.409238 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c6eb0-3bbd-4148-85b5-990fb9fc7fa7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:54:15.409892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c6eb0-3bbd-4148-85b5-990fb9fc7fa7 service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:54:15.410175 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c6eb0-3bbd-4148-85b5-990fb9fc7fa7 service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:54:15.410450 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c6eb0-3bbd-4148-85b5-990fb9fc7fa7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:54:15.412257 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4e6c6eb0-3bbd-4148-85b5-990fb9fc7fa7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:54:15.424009 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c6eb0-3bbd-4148-85b5-990fb9fc7fa7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:54:15.425584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c6eb0-3bbd-4148-85b5-990fb9fc7fa7 service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:54:15.436109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1406/2781] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:54:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:54:25.431067 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=58 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:54:52.788194 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:54:52.789696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:54:52.857251 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6b59d-1c94-49f5-9bfe-90e1cb770c14 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:54:52.860836 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9abbd-1245-4b20-aa8d-b5aa462b876f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:54:52.861475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9abbd-1245-4b20-aa8d-b5aa462b876f service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:54:52.861754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9abbd-1245-4b20-aa8d-b5aa462b876f service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:54:52.862024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9abbd-1245-4b20-aa8d-b5aa462b876f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:54:52.863788 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-62f9abbd-1245-4b20-aa8d-b5aa462b876f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:54:52.925765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9abbd-1245-4b20-aa8d-b5aa462b876f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:54:52.927124 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9abbd-1245-4b20-aa8d-b5aa462b876f service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:54:52.937417 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1376/2782] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:54:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:54:56.184321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c6eb0-3bbd-4148-85b5-990fb9fc7fa7 service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:54:56.185933 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4e6c6eb0-3bbd-4148-85b5-990fb9fc7fa7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:54:56.216055 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c6eb0-3bbd-4148-85b5-990fb9fc7fa7 service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:54:56.221142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760904b5-04ce-44e7-beee-8671a630fb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:54:56.222118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760904b5-04ce-44e7-beee-8671a630fb40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:54:56.222580 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760904b5-04ce-44e7-beee-8671a630fb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:54:56.222862 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760904b5-04ce-44e7-beee-8671a630fb40 admin admin] PATH_INFO: `/v3/users/7b4006c43f7944c083cf323db4df2b3b` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:54:56.224120 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760904b5-04ce-44e7-beee-8671a630fb40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b4006c43f7944c083cf323db4df2b3b)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:54:56.226239 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760904b5-04ce-44e7-beee-8671a630fb40 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:54:56.285669 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-760904b5-04ce-44e7-beee-8671a630fb40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b4006c43f7944c083cf323db4df2b3b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:54:56.286381 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-760904b5-04ce-44e7-beee-8671a630fb40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b4006c43f7944c083cf323db4df2b3b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:54:56.297782 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-760904b5-04ce-44e7-beee-8671a630fb40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b4006c43f7944c083cf323db4df2b3b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:54:56.313996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-760904b5-04ce-44e7-beee-8671a630fb40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b4006c43f7944c083cf323db4df2b3b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:54:56.321232 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-760904b5-04ce-44e7-beee-8671a630fb40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b4006c43f7944c083cf323db4df2b3b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:54:56.327668 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1407/2783] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:54:56 2022] DELETE /identity/v3/users/7b4006c43f7944c083cf323db4df2b3b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:54:56.507664 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9abbd-1245-4b20-aa8d-b5aa462b876f service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:54:56.519896 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9abbd-1245-4b20-aa8d-b5aa462b876f service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:54:56.581441 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851edb3-1ecd-4e73-95a1-2b2588a46ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:54:56.582213 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851edb3-1ecd-4e73-95a1-2b2588a46ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:54:56.582498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851edb3-1ecd-4e73-95a1-2b2588a46ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:54:56.582776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851edb3-1ecd-4e73-95a1-2b2588a46ce4 admin admin] PATH_INFO: `/v3/projects/8c46e98ae6174c1888cb10582f8a0bf4` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:54:56.583967 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851edb3-1ecd-4e73-95a1-2b2588a46ce4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c46e98ae6174c1888cb10582f8a0bf4)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:54:56.585902 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851edb3-1ecd-4e73-95a1-2b2588a46ce4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:54:56.632831 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9851edb3-1ecd-4e73-95a1-2b2588a46ce4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c46e98ae6174c1888cb10582f8a0bf4', 'request_id': 'req-9851edb3-1ecd-4e73-95a1-2b2588a46ce4'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:54:56.634804 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:54:56.640759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9851edb3-1ecd-4e73-95a1-2b2588a46ce4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c46e98ae6174c1888cb10582f8a0bf4', 'request_id': 'req-9851edb3-1ecd-4e73-95a1-2b2588a46ce4'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:54:56.648959 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9851edb3-1ecd-4e73-95a1-2b2588a46ce4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c46e98ae6174c1888cb10582f8a0bf4', 'request_id': 'req-9851edb3-1ecd-4e73-95a1-2b2588a46ce4'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:54:56.660212 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9851edb3-1ecd-4e73-95a1-2b2588a46ce4 admin admin] The token cache is being invalidate because project 8c46e98ae6174c1888cb10582f8a0bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:54:56.660500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9851edb3-1ecd-4e73-95a1-2b2588a46ce4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:54:56.661130 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9851edb3-1ecd-4e73-95a1-2b2588a46ce4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:54:56.662674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1377/2784] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:54:56 2022] DELETE /identity/v3/projects/8c46e98ae6174c1888cb10582f8a0bf4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:54:56.669358 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3c0dc-ef4a-424d-9348-2c524ce24c23 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:54:56.669657 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3c0dc-ef4a-424d-9348-2c524ce24c23 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:54:56.669945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3c0dc-ef4a-424d-9348-2c524ce24c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:54:56.688130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3ac3c0dc-ef4a-424d-9348-2c524ce24c23 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:54:56.689329 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3ac3c0dc-ef4a-424d-9348-2c524ce24c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:54:56.754009 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1408/2785] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:54:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:54:56.762100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679123cd-7ecb-4627-97b4-f168e0b07530 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:54:56.762462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679123cd-7ecb-4627-97b4-f168e0b07530 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:54:56.762655 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679123cd-7ecb-4627-97b4-f168e0b07530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:54:56.781616 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-679123cd-7ecb-4627-97b4-f168e0b07530 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:54:56.782841 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-679123cd-7ecb-4627-97b4-f168e0b07530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:54:56.790817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1378/2786] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:54:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:54:56.795424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3c0dc-ef4a-424d-9348-2c524ce24c23 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:54:56.805428 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3c0dc-ef4a-424d-9348-2c524ce24c23 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:54:56.810262 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d17009-60bf-4169-9a17-223fa2f0aa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:54:56.810919 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d17009-60bf-4169-9a17-223fa2f0aa48 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:54:56.811201 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d17009-60bf-4169-9a17-223fa2f0aa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:54:56.811478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d17009-60bf-4169-9a17-223fa2f0aa48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:54:56.812243 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d17009-60bf-4169-9a17-223fa2f0aa48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:54:56.812549 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d17009-60bf-4169-9a17-223fa2f0aa48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:54:56.814176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d17009-60bf-4169-9a17-223fa2f0aa48 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:54:56.826574 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1409/2787] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:54:56.831037 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679123cd-7ecb-4627-97b4-f168e0b07530 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:54:56.841332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679123cd-7ecb-4627-97b4-f168e0b07530 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:54:56.845924 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d49235-e0dc-4766-808f-9a2b41c21a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:54:56.846576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d49235-e0dc-4766-808f-9a2b41c21a13 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:54:56.846859 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d49235-e0dc-4766-808f-9a2b41c21a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:54:56.847136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d49235-e0dc-4766-808f-9a2b41c21a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:54:56.847821 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d49235-e0dc-4766-808f-9a2b41c21a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:54:56.849795 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d49235-e0dc-4766-808f-9a2b41c21a13 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:54:56.874858 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1379/2788] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:54:56 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:54:56.878985 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d17009-60bf-4169-9a17-223fa2f0aa48 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:54:56.890100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d17009-60bf-4169-9a17-223fa2f0aa48 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:54:56.947474 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e7395-cd6c-4b08-9331-cdaab9de5e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:54:56.948173 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e7395-cd6c-4b08-9331-cdaab9de5e79 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:54:56.948459 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e7395-cd6c-4b08-9331-cdaab9de5e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:54:56.949248 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e7395-cd6c-4b08-9331-cdaab9de5e79 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:54:56.949947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e7395-cd6c-4b08-9331-cdaab9de5e79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:54:56.951880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e7395-cd6c-4b08-9331-cdaab9de5e79 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:54:56.958426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:54:56.973780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1410/2789] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:54:56 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:54:56.978053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d49235-e0dc-4766-808f-9a2b41c21a13 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:54:56.988568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d49235-e0dc-4766-808f-9a2b41c21a13 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:54:56.993083 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ece63-487e-474f-a05f-52efd1093375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:54:56.993739 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ece63-487e-474f-a05f-52efd1093375 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:54:56.994023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ece63-487e-474f-a05f-52efd1093375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:54:56.994300 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ece63-487e-474f-a05f-52efd1093375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:54:56.995024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ece63-487e-474f-a05f-52efd1093375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:54:56.996780 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ece63-487e-474f-a05f-52efd1093375 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:54:57.007416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1380/2790] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:54:57.012560 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e7395-cd6c-4b08-9331-cdaab9de5e79 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:54:57.022586 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e7395-cd6c-4b08-9331-cdaab9de5e79 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:54:57.027842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae259f4-ba7f-43eb-8e87-45a06897f3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:54:57.028592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae259f4-ba7f-43eb-8e87-45a06897f3cf admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:54:57.028879 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae259f4-ba7f-43eb-8e87-45a06897f3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:54:57.029159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae259f4-ba7f-43eb-8e87-45a06897f3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:54:57.029889 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae259f4-ba7f-43eb-8e87-45a06897f3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:54:57.031487 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae259f4-ba7f-43eb-8e87-45a06897f3cf admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:54:57.041794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1411/2791] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:54:57.046642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ece63-487e-474f-a05f-52efd1093375 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:54:57.056615 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ece63-487e-474f-a05f-52efd1093375 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:54:57.061739 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfa708-3b32-47e7-b415-c0b463cd9833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:54:57.062639 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfa708-3b32-47e7-b415-c0b463cd9833 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:54:57.062907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfa708-3b32-47e7-b415-c0b463cd9833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:54:57.063180 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfa708-3b32-47e7-b415-c0b463cd9833 admin admin] PATH_INFO: `/v3/projects/b1ce15cfc6e54229b7a0e662cde8d913/users/3307799822b1435d9031c91a462d8a30/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:54:57.077945 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfa708-3b32-47e7-b415-c0b463cd9833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ce15cfc6e54229b7a0e662cde8d913, user_id=3307799822b1435d9031c91a462d8a30, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:54:57.080642 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfa708-3b32-47e7-b415-c0b463cd9833 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:54:57.094742 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1381/2792] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:54:57 2022] PUT /identity/v3/projects/b1ce15cfc6e54229b7a0e662cde8d913/users/3307799822b1435d9031c91a462d8a30/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:54:57.104869 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb55cf-3f2b-4279-8dad-7406c59a6c79 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:54:57.105792 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb55cf-3f2b-4279-8dad-7406c59a6c79 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:54:57.105792 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb55cf-3f2b-4279-8dad-7406c59a6c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:54:57.132836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f4fb55cf-3f2b-4279-8dad-7406c59a6c79 None None] MFA Rules not processed for user `3307799822b1435d9031c91a462d8a30`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:54:57.134098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f4fb55cf-3f2b-4279-8dad-7406c59a6c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:54:57.201320 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1412/2793] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:54:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:54:57.212045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfa708-3b32-47e7-b415-c0b463cd9833 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:54:57.213713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ecbfa708-3b32-47e7-b415-c0b463cd9833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:54:57.287848 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfa708-3b32-47e7-b415-c0b463cd9833 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:54:57.293966 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f08ed-53bf-4e6f-96c4-798718c3ef77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:54:57.294617 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f08ed-53bf-4e6f-96c4-798718c3ef77 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:54:57.295223 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f08ed-53bf-4e6f-96c4-798718c3ef77 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:54:57.295505 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f08ed-53bf-4e6f-96c4-798718c3ef77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:54:57.303359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f08ed-53bf-4e6f-96c4-798718c3ef77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:54:57.305007 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f08ed-53bf-4e6f-96c4-798718c3ef77 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:54:57.314727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1382/2794] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:54:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:55:06.847385 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb55cf-3f2b-4279-8dad-7406c59a6c79 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:06.917074 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb55cf-3f2b-4279-8dad-7406c59a6c79 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:06.922389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc146298-74e8-497e-a977-b2d1c89d7f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:06.923270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc146298-74e8-497e-a977-b2d1c89d7f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:06.923551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc146298-74e8-497e-a977-b2d1c89d7f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:06.923830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc146298-74e8-497e-a977-b2d1c89d7f9e admin admin] PATH_INFO: `/v3/users/3307799822b1435d9031c91a462d8a30` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:06.925077 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc146298-74e8-497e-a977-b2d1c89d7f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3307799822b1435d9031c91a462d8a30)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:06.927032 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc146298-74e8-497e-a977-b2d1c89d7f9e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:06.982733 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dc146298-74e8-497e-a977-b2d1c89d7f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3307799822b1435d9031c91a462d8a30'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:06.983511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dc146298-74e8-497e-a977-b2d1c89d7f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3307799822b1435d9031c91a462d8a30'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:06.994326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dc146298-74e8-497e-a977-b2d1c89d7f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3307799822b1435d9031c91a462d8a30'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:07.011659 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:07.013236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dc146298-74e8-497e-a977-b2d1c89d7f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3307799822b1435d9031c91a462d8a30'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:07.024911 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dc146298-74e8-497e-a977-b2d1c89d7f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3307799822b1435d9031c91a462d8a30'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:07.034533 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1413/2795] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:55:06 2022] DELETE /identity/v3/users/3307799822b1435d9031c91a462d8a30 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:07.038493 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee666ed-5599-4dd4-84b7-29b473e1a457 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:07.039174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee666ed-5599-4dd4-84b7-29b473e1a457 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:07.039174 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee666ed-5599-4dd4-84b7-29b473e1a457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:07.058857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1ee666ed-5599-4dd4-84b7-29b473e1a457 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:07.060357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1ee666ed-5599-4dd4-84b7-29b473e1a457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:07.128040 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1383/2796] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:07.139285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc146298-74e8-497e-a977-b2d1c89d7f9e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:07.140885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-dc146298-74e8-497e-a977-b2d1c89d7f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:07.203034 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc146298-74e8-497e-a977-b2d1c89d7f9e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:07.207133 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56935cb-8d5c-402d-aefc-db57c4e78303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:07.207810 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56935cb-8d5c-402d-aefc-db57c4e78303 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:07.208130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56935cb-8d5c-402d-aefc-db57c4e78303 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:07.208402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56935cb-8d5c-402d-aefc-db57c4e78303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:07.218285 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56935cb-8d5c-402d-aefc-db57c4e78303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:07.219743 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56935cb-8d5c-402d-aefc-db57c4e78303 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:07.230641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1414/2797] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:55:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:55:07.545204 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee666ed-5599-4dd4-84b7-29b473e1a457 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:07.546704 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1ee666ed-5599-4dd4-84b7-29b473e1a457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:07.553874 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:07.563129 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee666ed-5599-4dd4-84b7-29b473e1a457 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:07.570173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c8b5e-b200-4fd7-bc77-7b72b41a934a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:07.571276 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c8b5e-b200-4fd7-bc77-7b72b41a934a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:07.571764 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c8b5e-b200-4fd7-bc77-7b72b41a934a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:07.572156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c8b5e-b200-4fd7-bc77-7b72b41a934a admin admin] PATH_INFO: `/v3/projects/b1ce15cfc6e54229b7a0e662cde8d913` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:07.573702 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c8b5e-b200-4fd7-bc77-7b72b41a934a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1ce15cfc6e54229b7a0e662cde8d913)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:07.576092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c8b5e-b200-4fd7-bc77-7b72b41a934a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:07.625410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd4c8b5e-b200-4fd7-bc77-7b72b41a934a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1ce15cfc6e54229b7a0e662cde8d913', 'request_id': 'req-cd4c8b5e-b200-4fd7-bc77-7b72b41a934a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:07.631652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd4c8b5e-b200-4fd7-bc77-7b72b41a934a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1ce15cfc6e54229b7a0e662cde8d913', 'request_id': 'req-cd4c8b5e-b200-4fd7-bc77-7b72b41a934a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:07.639457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd4c8b5e-b200-4fd7-bc77-7b72b41a934a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1ce15cfc6e54229b7a0e662cde8d913', 'request_id': 'req-cd4c8b5e-b200-4fd7-bc77-7b72b41a934a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:07.650192 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd4c8b5e-b200-4fd7-bc77-7b72b41a934a admin admin] The token cache is being invalidate because project b1ce15cfc6e54229b7a0e662cde8d913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:55:07.650480 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd4c8b5e-b200-4fd7-bc77-7b72b41a934a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:07.651101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd4c8b5e-b200-4fd7-bc77-7b72b41a934a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:07.653207 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1384/2798] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:55:07 2022] DELETE /identity/v3/projects/b1ce15cfc6e54229b7a0e662cde8d913 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:55:07.658245 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce943ef-b6e0-4f61-9af6-ee2430be064b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:07.658542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce943ef-b6e0-4f61-9af6-ee2430be064b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:07.658825 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce943ef-b6e0-4f61-9af6-ee2430be064b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:07.677389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6ce943ef-b6e0-4f61-9af6-ee2430be064b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:07.678621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6ce943ef-b6e0-4f61-9af6-ee2430be064b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:07.746016 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1415/2799] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:07.755051 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fb021-214c-429a-95a0-1de620206f73 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:07.755363 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fb021-214c-429a-95a0-1de620206f73 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:07.755648 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fb021-214c-429a-95a0-1de620206f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:07.774223 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-520fb021-214c-429a-95a0-1de620206f73 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:07.775438 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-520fb021-214c-429a-95a0-1de620206f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:07.784287 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1385/2800] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:07.788505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce943ef-b6e0-4f61-9af6-ee2430be064b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:07.799151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce943ef-b6e0-4f61-9af6-ee2430be064b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:07.803734 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa3483-4d0b-46ef-b518-7d5a442052c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:07.804427 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa3483-4d0b-46ef-b518-7d5a442052c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:07.804719 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa3483-4d0b-46ef-b518-7d5a442052c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:07.804999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa3483-4d0b-46ef-b518-7d5a442052c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:07.805726 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa3483-4d0b-46ef-b518-7d5a442052c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:55:07.806035 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa3483-4d0b-46ef-b518-7d5a442052c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:07.807641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa3483-4d0b-46ef-b518-7d5a442052c8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:07.819321 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1416/2801] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:55:07.824291 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fb021-214c-429a-95a0-1de620206f73 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:07.833731 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fb021-214c-429a-95a0-1de620206f73 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:07.838562 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3100f-e2b3-4565-b0b4-62f5891d93d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:07.840153 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3100f-e2b3-4565-b0b4-62f5891d93d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:07.840447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3100f-e2b3-4565-b0b4-62f5891d93d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:07.840754 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3100f-e2b3-4565-b0b4-62f5891d93d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:07.841451 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3100f-e2b3-4565-b0b4-62f5891d93d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:07.843371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3100f-e2b3-4565-b0b4-62f5891d93d1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:07.869938 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1386/2802] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:55:07 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:55:07.874845 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa3483-4d0b-46ef-b518-7d5a442052c8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:07.885679 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa3483-4d0b-46ef-b518-7d5a442052c8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:07.944847 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25e9e7-39e3-4241-b491-b38860b7c30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:07.945511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25e9e7-39e3-4241-b491-b38860b7c30a admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:07.945792 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25e9e7-39e3-4241-b491-b38860b7c30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:07.946100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25e9e7-39e3-4241-b491-b38860b7c30a admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:07.946770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25e9e7-39e3-4241-b491-b38860b7c30a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:07.948741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25e9e7-39e3-4241-b491-b38860b7c30a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:07.969794 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1417/2803] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:55:07 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:55:07.973924 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3100f-e2b3-4565-b0b4-62f5891d93d1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:07.984904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3100f-e2b3-4565-b0b4-62f5891d93d1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:07.989643 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac411b8-63e7-43c3-8e9b-5911524d47dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:07.990303 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac411b8-63e7-43c3-8e9b-5911524d47dc admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:07.990619 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac411b8-63e7-43c3-8e9b-5911524d47dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:07.990900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac411b8-63e7-43c3-8e9b-5911524d47dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:07.991626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac411b8-63e7-43c3-8e9b-5911524d47dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:07.993288 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac411b8-63e7-43c3-8e9b-5911524d47dc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:08.004451 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1387/2804] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:55:08.008872 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25e9e7-39e3-4241-b491-b38860b7c30a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:08.019391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25e9e7-39e3-4241-b491-b38860b7c30a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:08.024146 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d1d83-f21a-4b41-9de3-082985c71c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:08.024893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d1d83-f21a-4b41-9de3-082985c71c23 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:08.025209 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d1d83-f21a-4b41-9de3-082985c71c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:08.025492 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d1d83-f21a-4b41-9de3-082985c71c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:08.026223 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d1d83-f21a-4b41-9de3-082985c71c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:08.028490 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d1d83-f21a-4b41-9de3-082985c71c23 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:08.029882 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:08.041404 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1418/2805] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:55:08.045997 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac411b8-63e7-43c3-8e9b-5911524d47dc admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:08.061890 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac411b8-63e7-43c3-8e9b-5911524d47dc admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:08.066883 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cba1e5-1052-4cdc-893b-71336ff963eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:08.067790 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cba1e5-1052-4cdc-893b-71336ff963eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:08.068118 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cba1e5-1052-4cdc-893b-71336ff963eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:08.068546 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cba1e5-1052-4cdc-893b-71336ff963eb admin admin] PATH_INFO: `/v3/projects/e531c93fb51e4647af1c7827a336f1cc/users/3394636668b34a66958df22cee9a3088/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:08.085116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cba1e5-1052-4cdc-893b-71336ff963eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e531c93fb51e4647af1c7827a336f1cc, user_id=3394636668b34a66958df22cee9a3088, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:08.087812 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cba1e5-1052-4cdc-893b-71336ff963eb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:08.101768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1388/2806] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:55:08 2022] PUT /identity/v3/projects/e531c93fb51e4647af1c7827a336f1cc/users/3394636668b34a66958df22cee9a3088/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:08.163802 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692b74c-eace-41e4-8900-674463dfe33b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:08.164197 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692b74c-eace-41e4-8900-674463dfe33b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:08.164483 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692b74c-eace-41e4-8900-674463dfe33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:08.194443 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b692b74c-eace-41e4-8900-674463dfe33b None None] MFA Rules not processed for user `3394636668b34a66958df22cee9a3088`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:08.195726 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b692b74c-eace-41e4-8900-674463dfe33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:08.270871 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1419/2807] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:08.283061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cba1e5-1052-4cdc-893b-71336ff963eb admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:08.284865 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f4cba1e5-1052-4cdc-893b-71336ff963eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:08.366084 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cba1e5-1052-4cdc-893b-71336ff963eb admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:08.372374 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f4191-900d-4d36-a04f-77e71d0da7ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c48d41dcf9b344a8b2fec77f4dd75fcb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:08.373022 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f4191-900d-4d36-a04f-77e71d0da7ab service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:08.373301 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f4191-900d-4d36-a04f-77e71d0da7ab service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:08.373576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f4191-900d-4d36-a04f-77e71d0da7ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:08.381952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f4191-900d-4d36-a04f-77e71d0da7ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:08.383563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f4191-900d-4d36-a04f-77e71d0da7ab service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:08.394519 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1389/2808] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:55:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 09:55:11.066472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692b74c-eace-41e4-8900-674463dfe33b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:11.140335 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692b74c-eace-41e4-8900-674463dfe33b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:11.144957 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f26c7-55a0-45d1-8bcf-64b4d84a6b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:11.145863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f26c7-55a0-45d1-8bcf-64b4d84a6b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:11.146170 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f26c7-55a0-45d1-8bcf-64b4d84a6b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:11.146429 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f26c7-55a0-45d1-8bcf-64b4d84a6b28 admin admin] PATH_INFO: `/v3/users/3394636668b34a66958df22cee9a3088` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:11.147639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f26c7-55a0-45d1-8bcf-64b4d84a6b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3394636668b34a66958df22cee9a3088)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:11.149653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f26c7-55a0-45d1-8bcf-64b4d84a6b28 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:11.200536 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7a1f26c7-55a0-45d1-8bcf-64b4d84a6b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3394636668b34a66958df22cee9a3088'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:11.201194 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7a1f26c7-55a0-45d1-8bcf-64b4d84a6b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3394636668b34a66958df22cee9a3088'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:11.208911 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7a1f26c7-55a0-45d1-8bcf-64b4d84a6b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3394636668b34a66958df22cee9a3088'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:11.221888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7a1f26c7-55a0-45d1-8bcf-64b4d84a6b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3394636668b34a66958df22cee9a3088'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:11.230195 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7a1f26c7-55a0-45d1-8bcf-64b4d84a6b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3394636668b34a66958df22cee9a3088'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:11.236534 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1420/2809] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:55:11 2022] DELETE /identity/v3/users/3394636668b34a66958df22cee9a3088 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:11.241130 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78abb2-4193-4141-b8a0-cf19e8e4ef82 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:11.241696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78abb2-4193-4141-b8a0-cf19e8e4ef82 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:11.241874 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78abb2-4193-4141-b8a0-cf19e8e4ef82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:11.254703 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:11.262629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2b78abb2-4193-4141-b8a0-cf19e8e4ef82 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:11.264524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2b78abb2-4193-4141-b8a0-cf19e8e4ef82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:11.328388 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1390/2810] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:11.338346 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f26c7-55a0-45d1-8bcf-64b4d84a6b28 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:11.340237 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7a1f26c7-55a0-45d1-8bcf-64b4d84a6b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:11.347360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:11.404119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f26c7-55a0-45d1-8bcf-64b4d84a6b28 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:11.407896 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97369260-b8a0-48d2-98a6-7bc56df33afa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:11.408559 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97369260-b8a0-48d2-98a6-7bc56df33afa service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:11.408843 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97369260-b8a0-48d2-98a6-7bc56df33afa service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:11.409123 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97369260-b8a0-48d2-98a6-7bc56df33afa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:11.419831 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97369260-b8a0-48d2-98a6-7bc56df33afa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:11.421317 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97369260-b8a0-48d2-98a6-7bc56df33afa service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:11.432546 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1421/2811] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:55:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:55:11.721094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78abb2-4193-4141-b8a0-cf19e8e4ef82 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:11.721094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2b78abb2-4193-4141-b8a0-cf19e8e4ef82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:11.739857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78abb2-4193-4141-b8a0-cf19e8e4ef82 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:11.746034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657467c-aff1-4638-8a0f-7aecac69a235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:11.747193 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657467c-aff1-4638-8a0f-7aecac69a235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:11.747492 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657467c-aff1-4638-8a0f-7aecac69a235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:11.747799 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657467c-aff1-4638-8a0f-7aecac69a235 admin admin] PATH_INFO: `/v3/projects/e531c93fb51e4647af1c7827a336f1cc` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:11.749107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657467c-aff1-4638-8a0f-7aecac69a235 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e531c93fb51e4647af1c7827a336f1cc)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:11.751200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657467c-aff1-4638-8a0f-7aecac69a235 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:11.801402 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7657467c-aff1-4638-8a0f-7aecac69a235 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e531c93fb51e4647af1c7827a336f1cc', 'request_id': 'req-7657467c-aff1-4638-8a0f-7aecac69a235'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:11.807830 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7657467c-aff1-4638-8a0f-7aecac69a235 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e531c93fb51e4647af1c7827a336f1cc', 'request_id': 'req-7657467c-aff1-4638-8a0f-7aecac69a235'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:11.816143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7657467c-aff1-4638-8a0f-7aecac69a235 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e531c93fb51e4647af1c7827a336f1cc', 'request_id': 'req-7657467c-aff1-4638-8a0f-7aecac69a235'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:11.830237 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7657467c-aff1-4638-8a0f-7aecac69a235 admin admin] The token cache is being invalidate because project e531c93fb51e4647af1c7827a336f1cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:55:11.830555 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7657467c-aff1-4638-8a0f-7aecac69a235 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:11.831270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7657467c-aff1-4638-8a0f-7aecac69a235 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:11.832924 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1391/2812] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:55:11 2022] DELETE /identity/v3/projects/e531c93fb51e4647af1c7827a336f1cc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:11.838614 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb7a54-0e8a-46fd-96cf-26164aa637e7 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:11.839907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb7a54-0e8a-46fd-96cf-26164aa637e7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:11.840284 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb7a54-0e8a-46fd-96cf-26164aa637e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:11.860340 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-42eb7a54-0e8a-46fd-96cf-26164aa637e7 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:11.861561 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-42eb7a54-0e8a-46fd-96cf-26164aa637e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:11.924622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1422/2813] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:11.933372 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c96049-2dbc-491c-ba09-4584872930b1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:11.933594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c96049-2dbc-491c-ba09-4584872930b1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:11.933949 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c96049-2dbc-491c-ba09-4584872930b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:11.952538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-49c96049-2dbc-491c-ba09-4584872930b1 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:11.953761 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-49c96049-2dbc-491c-ba09-4584872930b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:11.961733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1392/2814] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:11.966095 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb7a54-0e8a-46fd-96cf-26164aa637e7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:11.976091 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb7a54-0e8a-46fd-96cf-26164aa637e7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:11.981426 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fecc0-e58c-42d5-967c-9216d3993c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:11.982088 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fecc0-e58c-42d5-967c-9216d3993c7d admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:11.982373 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fecc0-e58c-42d5-967c-9216d3993c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:11.982684 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fecc0-e58c-42d5-967c-9216d3993c7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:11.983412 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fecc0-e58c-42d5-967c-9216d3993c7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:55:11.983721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fecc0-e58c-42d5-967c-9216d3993c7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:11.985369 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fecc0-e58c-42d5-967c-9216d3993c7d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:11.997795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1423/2815] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:55:12.002220 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c96049-2dbc-491c-ba09-4584872930b1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:12.011899 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c96049-2dbc-491c-ba09-4584872930b1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:12.016721 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45444719-cc78-48af-add2-70ec5907b70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:12.017385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45444719-cc78-48af-add2-70ec5907b70a admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:12.017672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45444719-cc78-48af-add2-70ec5907b70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:12.017950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45444719-cc78-48af-add2-70ec5907b70a admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:12.018669 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45444719-cc78-48af-add2-70ec5907b70a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:12.020649 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45444719-cc78-48af-add2-70ec5907b70a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:12.050717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1393/2816] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:55:12 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:55:12.055198 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fecc0-e58c-42d5-967c-9216d3993c7d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:12.066569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fecc0-e58c-42d5-967c-9216d3993c7d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:12.131512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ff5cf-1724-4601-8839-4b50d168bf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:12.132225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ff5cf-1724-4601-8839-4b50d168bf3d admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:12.132511 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ff5cf-1724-4601-8839-4b50d168bf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:12.132783 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ff5cf-1724-4601-8839-4b50d168bf3d admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:12.133510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ff5cf-1724-4601-8839-4b50d168bf3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:12.135446 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ff5cf-1724-4601-8839-4b50d168bf3d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:12.157275 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1424/2817] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:55:12 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:55:12.163398 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45444719-cc78-48af-add2-70ec5907b70a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:12.175886 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45444719-cc78-48af-add2-70ec5907b70a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:12.180662 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b9690-0ffd-4dae-959e-47258afe7c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:12.181353 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b9690-0ffd-4dae-959e-47258afe7c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:12.181631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b9690-0ffd-4dae-959e-47258afe7c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:12.181910 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b9690-0ffd-4dae-959e-47258afe7c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:12.182700 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b9690-0ffd-4dae-959e-47258afe7c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:12.184392 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b9690-0ffd-4dae-959e-47258afe7c7b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:12.195588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1394/2818] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:55:12.199807 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ff5cf-1724-4601-8839-4b50d168bf3d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:12.210177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ff5cf-1724-4601-8839-4b50d168bf3d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:12.215972 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a1440-5eea-4035-ad78-841258825497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:12.216653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a1440-5eea-4035-ad78-841258825497 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:12.216943 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a1440-5eea-4035-ad78-841258825497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:12.217221 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a1440-5eea-4035-ad78-841258825497 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:12.217960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a1440-5eea-4035-ad78-841258825497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:12.219584 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a1440-5eea-4035-ad78-841258825497 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:12.231340 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1425/2819] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:55:12.235631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b9690-0ffd-4dae-959e-47258afe7c7b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:12.246516 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b9690-0ffd-4dae-959e-47258afe7c7b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:12.252062 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6deb9-91ad-4c15-ba0f-89e75f62cbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:12.252953 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6deb9-91ad-4c15-ba0f-89e75f62cbf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:12.253242 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6deb9-91ad-4c15-ba0f-89e75f62cbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:12.253520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6deb9-91ad-4c15-ba0f-89e75f62cbf7 admin admin] PATH_INFO: `/v3/projects/e3f5b2cac22a4c478f0d8331891dd480/users/e1520cbbf7984abfa45ffcbdf3714d0e/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:12.271887 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6deb9-91ad-4c15-ba0f-89e75f62cbf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3f5b2cac22a4c478f0d8331891dd480, user_id=e1520cbbf7984abfa45ffcbdf3714d0e, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:12.275019 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6deb9-91ad-4c15-ba0f-89e75f62cbf7 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:12.291817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1395/2820] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:55:12 2022] PUT /identity/v3/projects/e3f5b2cac22a4c478f0d8331891dd480/users/e1520cbbf7984abfa45ffcbdf3714d0e/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:12.304441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e4350-faad-4f43-b5f7-2831c200bb4f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:12.304741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e4350-faad-4f43-b5f7-2831c200bb4f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:12.305019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e4350-faad-4f43-b5f7-2831c200bb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:12.330003 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:12.333367 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c10e4350-faad-4f43-b5f7-2831c200bb4f None None] MFA Rules not processed for user `e1520cbbf7984abfa45ffcbdf3714d0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:12.335019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c10e4350-faad-4f43-b5f7-2831c200bb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:12.416462 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1426/2821] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:12.423396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6deb9-91ad-4c15-ba0f-89e75f62cbf7 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:12.454904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:12.490015 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6deb9-91ad-4c15-ba0f-89e75f62cbf7 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:12.494611 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e468bb3-c1d0-4a6d-b835-b5e3751814cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:12.495378 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e468bb3-c1d0-4a6d-b835-b5e3751814cd admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:12.495707 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e468bb3-c1d0-4a6d-b835-b5e3751814cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:12.495965 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e468bb3-c1d0-4a6d-b835-b5e3751814cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:12.496667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e468bb3-c1d0-4a6d-b835-b5e3751814cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:12.498590 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e468bb3-c1d0-4a6d-b835-b5e3751814cd admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:12.525973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1396/2822] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:55:12 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:55:12.530775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e4350-faad-4f43-b5f7-2831c200bb4f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:12.542000 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e4350-faad-4f43-b5f7-2831c200bb4f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:12.601960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041aa5c-1ab4-4d04-8134-b5633abddf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:12.602647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041aa5c-1ab4-4d04-8134-b5633abddf9d admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:12.602934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041aa5c-1ab4-4d04-8134-b5633abddf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:12.603213 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041aa5c-1ab4-4d04-8134-b5633abddf9d admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:12.603934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041aa5c-1ab4-4d04-8134-b5633abddf9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:12.606016 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041aa5c-1ab4-4d04-8134-b5633abddf9d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:12.627431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1427/2823] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:55:12 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:55:12.632757 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e468bb3-c1d0-4a6d-b835-b5e3751814cd admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:12.644723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e468bb3-c1d0-4a6d-b835-b5e3751814cd admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:12.651032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a7e9e-0146-45a1-acd5-d1f8ef7c1501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:12.652515 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a7e9e-0146-45a1-acd5-d1f8ef7c1501 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:12.653077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a7e9e-0146-45a1-acd5-d1f8ef7c1501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:12.653304 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a7e9e-0146-45a1-acd5-d1f8ef7c1501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:12.654044 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a7e9e-0146-45a1-acd5-d1f8ef7c1501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:12.655700 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a7e9e-0146-45a1-acd5-d1f8ef7c1501 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:12.668147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1397/2824] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:55:12.672463 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041aa5c-1ab4-4d04-8134-b5633abddf9d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:12.683592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041aa5c-1ab4-4d04-8134-b5633abddf9d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:12.688438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13679c0b-de20-44e9-9a1a-0efd817340ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:12.689340 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13679c0b-de20-44e9-9a1a-0efd817340ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:12.689618 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13679c0b-de20-44e9-9a1a-0efd817340ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:12.689890 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13679c0b-de20-44e9-9a1a-0efd817340ca admin admin] PATH_INFO: `/v3/projects/9a7a3e7442784c17ac6136bd497e4680/users/3d5e96c29e4847f38da3fd64df4b1563/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:12.706830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13679c0b-de20-44e9-9a1a-0efd817340ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7a3e7442784c17ac6136bd497e4680, user_id=3d5e96c29e4847f38da3fd64df4b1563, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:12.709609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13679c0b-de20-44e9-9a1a-0efd817340ca admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:12.726305 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1428/2825] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:55:12 2022] PUT /identity/v3/projects/9a7a3e7442784c17ac6136bd497e4680/users/3d5e96c29e4847f38da3fd64df4b1563/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:55:12.730458 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a7e9e-0146-45a1-acd5-d1f8ef7c1501 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:12.745478 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a7e9e-0146-45a1-acd5-d1f8ef7c1501 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:12.810346 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142e320-8f7f-46ce-9696-210b007157af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:12.811011 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142e320-8f7f-46ce-9696-210b007157af admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:12.811292 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142e320-8f7f-46ce-9696-210b007157af admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:12.811566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142e320-8f7f-46ce-9696-210b007157af admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:12.812318 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e142e320-8f7f-46ce-9696-210b007157af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:12.813930 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e142e320-8f7f-46ce-9696-210b007157af admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:12.824368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1398/2826] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:55:12.828770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13679c0b-de20-44e9-9a1a-0efd817340ca admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:12.839789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13679c0b-de20-44e9-9a1a-0efd817340ca admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:12.844700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da2676-2646-41c0-82be-2fa07d031f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:12.845578 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da2676-2646-41c0-82be-2fa07d031f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:12.845855 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da2676-2646-41c0-82be-2fa07d031f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:12.846130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da2676-2646-41c0-82be-2fa07d031f11 admin admin] PATH_INFO: `/v3/projects/9a7a3e7442784c17ac6136bd497e4680/users/3d5e96c29e4847f38da3fd64df4b1563/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:12.848106 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da2676-2646-41c0-82be-2fa07d031f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7a3e7442784c17ac6136bd497e4680, user_id=3d5e96c29e4847f38da3fd64df4b1563, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:12.850739 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da2676-2646-41c0-82be-2fa07d031f11 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:12.861210 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1429/2827] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:55:12 2022] PUT /identity/v3/projects/9a7a3e7442784c17ac6136bd497e4680/users/3d5e96c29e4847f38da3fd64df4b1563/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:55:12.872045 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2cba9-322f-4a9f-a2c2-cfe9330b53b0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:12.872341 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2cba9-322f-4a9f-a2c2-cfe9330b53b0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:12.872646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2cba9-322f-4a9f-a2c2-cfe9330b53b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:12.901024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-dee2cba9-322f-4a9f-a2c2-cfe9330b53b0 None None] MFA Rules not processed for user `3d5e96c29e4847f38da3fd64df4b1563`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:12.902471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dee2cba9-322f-4a9f-a2c2-cfe9330b53b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:12.974221 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1399/2828] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:12.985964 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da2676-2646-41c0-82be-2fa07d031f11 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:12.987698 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-44da2676-2646-41c0-82be-2fa07d031f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:13.059501 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da2676-2646-41c0-82be-2fa07d031f11 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:13.063380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32ec5b-6e6a-4190-afe3-a2683305d571 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:13.064079 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32ec5b-6e6a-4190-afe3-a2683305d571 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:13.064368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32ec5b-6e6a-4190-afe3-a2683305d571 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:13.064645 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32ec5b-6e6a-4190-afe3-a2683305d571 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:13.075102 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:13.138168 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32ec5b-6e6a-4190-afe3-a2683305d571 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:13.139892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32ec5b-6e6a-4190-afe3-a2683305d571 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:13.150830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1430/2829] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:55:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:55:19.963088 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2cba9-322f-4a9f-a2c2-cfe9330b53b0 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:19.972858 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2cba9-322f-4a9f-a2c2-cfe9330b53b0 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:19.976356 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1f303-e2bc-4c44-a4a8-51223bed682b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:19.976996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1f303-e2bc-4c44-a4a8-51223bed682b service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:19.977280 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1f303-e2bc-4c44-a4a8-51223bed682b service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:19.977551 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1f303-e2bc-4c44-a4a8-51223bed682b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:19.988261 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1f303-e2bc-4c44-a4a8-51223bed682b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:19.989728 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1f303-e2bc-4c44-a4a8-51223bed682b service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:20.000710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1400/2830] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:55:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:55:23.145982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=23 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:29.995421 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:49.014851 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32ec5b-6e6a-4190-afe3-a2683305d571 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:49.077319 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32ec5b-6e6a-4190-afe3-a2683305d571 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:49.081770 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a2b33-9fce-4ac3-9301-6a661c2548f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:49.082630 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a2b33-9fce-4ac3-9301-6a661c2548f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:49.082918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a2b33-9fce-4ac3-9301-6a661c2548f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:49.083192 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a2b33-9fce-4ac3-9301-6a661c2548f0 admin admin] PATH_INFO: `/v3/users/e1520cbbf7984abfa45ffcbdf3714d0e` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:49.084388 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a2b33-9fce-4ac3-9301-6a661c2548f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1520cbbf7984abfa45ffcbdf3714d0e)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:49.086314 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a2b33-9fce-4ac3-9301-6a661c2548f0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:49.130791 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-cf9a2b33-9fce-4ac3-9301-6a661c2548f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1520cbbf7984abfa45ffcbdf3714d0e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:49.131429 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-cf9a2b33-9fce-4ac3-9301-6a661c2548f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1520cbbf7984abfa45ffcbdf3714d0e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:49.138443 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-cf9a2b33-9fce-4ac3-9301-6a661c2548f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1520cbbf7984abfa45ffcbdf3714d0e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:49.150901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-cf9a2b33-9fce-4ac3-9301-6a661c2548f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1520cbbf7984abfa45ffcbdf3714d0e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:49.157751 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-cf9a2b33-9fce-4ac3-9301-6a661c2548f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1520cbbf7984abfa45ffcbdf3714d0e'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:49.163760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1431/2831] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:55:49 2022] DELETE /identity/v3/users/e1520cbbf7984abfa45ffcbdf3714d0e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:55:49.168733 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6977c-c307-43f8-a2b8-2ac8b55b4abb None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:49.169210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6977c-c307-43f8-a2b8-2ac8b55b4abb None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:49.169308 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec6977c-c307-43f8-a2b8-2ac8b55b4abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:49.187751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1ec6977c-c307-43f8-a2b8-2ac8b55b4abb None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:49.188990 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1ec6977c-c307-43f8-a2b8-2ac8b55b4abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:49.247806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1401/2832] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:49.258781 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a2b33-9fce-4ac3-9301-6a661c2548f0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:49.260256 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cf9a2b33-9fce-4ac3-9301-6a661c2548f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:49.320871 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a2b33-9fce-4ac3-9301-6a661c2548f0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:49.324465 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55aa15-6c26-4a4d-88eb-a67684d8a524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:49.325112 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55aa15-6c26-4a4d-88eb-a67684d8a524 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:49.325393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55aa15-6c26-4a4d-88eb-a67684d8a524 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:49.325667 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55aa15-6c26-4a4d-88eb-a67684d8a524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:49.335780 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55aa15-6c26-4a4d-88eb-a67684d8a524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:49.337273 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55aa15-6c26-4a4d-88eb-a67684d8a524 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:49.348111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1432/2833] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:55:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:55:49.641060 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6977c-c307-43f8-a2b8-2ac8b55b4abb None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:49.642672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1ec6977c-c307-43f8-a2b8-2ac8b55b4abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:49.658383 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec6977c-c307-43f8-a2b8-2ac8b55b4abb None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:49.663316 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:49.664195 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:49.664483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:49.664763 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672 admin admin] PATH_INFO: `/v3/projects/e3f5b2cac22a4c478f0d8331891dd480` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:49.665963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3f5b2cac22a4c478f0d8331891dd480)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:49.668366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:49.718500 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3f5b2cac22a4c478f0d8331891dd480', 'request_id': 'req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:49.725432 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3f5b2cac22a4c478f0d8331891dd480', 'request_id': 'req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:49.733589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3f5b2cac22a4c478f0d8331891dd480', 'request_id': 'req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:49.745103 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672 admin admin] The token cache is being invalidate because project e3f5b2cac22a4c478f0d8331891dd480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:55:49.745464 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:49.746036 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:49.747605 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1402/2834] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:55:49 2022] DELETE /identity/v3/projects/e3f5b2cac22a4c478f0d8331891dd480 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:49.752178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55aa15-6c26-4a4d-88eb-a67684d8a524 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:49.762186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55aa15-6c26-4a4d-88eb-a67684d8a524 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:49.782568 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:49.823068 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762c7ba-83e3-4384-82dc-849db2d45694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:49.824024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762c7ba-83e3-4384-82dc-849db2d45694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:49.824313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762c7ba-83e3-4384-82dc-849db2d45694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:49.824590 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762c7ba-83e3-4384-82dc-849db2d45694 admin admin] PATH_INFO: `/v3/users/3d5e96c29e4847f38da3fd64df4b1563` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:49.825766 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762c7ba-83e3-4384-82dc-849db2d45694 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d5e96c29e4847f38da3fd64df4b1563)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:49.827712 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762c7ba-83e3-4384-82dc-849db2d45694 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:49.874136 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f762c7ba-83e3-4384-82dc-849db2d45694 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d5e96c29e4847f38da3fd64df4b1563'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:49.874775 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f762c7ba-83e3-4384-82dc-849db2d45694 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d5e96c29e4847f38da3fd64df4b1563'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:49.881164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f762c7ba-83e3-4384-82dc-849db2d45694 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d5e96c29e4847f38da3fd64df4b1563'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:49.893598 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f762c7ba-83e3-4384-82dc-849db2d45694 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d5e96c29e4847f38da3fd64df4b1563'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:49.900593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f762c7ba-83e3-4384-82dc-849db2d45694 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d5e96c29e4847f38da3fd64df4b1563'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:49.907366 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1433/2835] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:55:49 2022] DELETE /identity/v3/users/3d5e96c29e4847f38da3fd64df4b1563 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:50.176669 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:50.178552 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:50.193122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833b8c1-b6db-41cd-a5c3-ed3a83ea7672 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:50.251669 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be262f-836c-4728-b39e-5d7a4942cabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:50.252599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be262f-836c-4728-b39e-5d7a4942cabe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:50.252884 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be262f-836c-4728-b39e-5d7a4942cabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:50.253163 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be262f-836c-4728-b39e-5d7a4942cabe admin admin] PATH_INFO: `/v3/projects/9a7a3e7442784c17ac6136bd497e4680` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:50.254347 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be262f-836c-4728-b39e-5d7a4942cabe admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a7a3e7442784c17ac6136bd497e4680)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:50.256399 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be262f-836c-4728-b39e-5d7a4942cabe admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:50.287137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:50.306215 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-88be262f-836c-4728-b39e-5d7a4942cabe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a7a3e7442784c17ac6136bd497e4680', 'request_id': 'req-88be262f-836c-4728-b39e-5d7a4942cabe'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:50.313101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-88be262f-836c-4728-b39e-5d7a4942cabe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a7a3e7442784c17ac6136bd497e4680', 'request_id': 'req-88be262f-836c-4728-b39e-5d7a4942cabe'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:50.320567 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-88be262f-836c-4728-b39e-5d7a4942cabe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a7a3e7442784c17ac6136bd497e4680', 'request_id': 'req-88be262f-836c-4728-b39e-5d7a4942cabe'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:50.331855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-88be262f-836c-4728-b39e-5d7a4942cabe admin admin] The token cache is being invalidate because project 9a7a3e7442784c17ac6136bd497e4680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:55:50.332182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-88be262f-836c-4728-b39e-5d7a4942cabe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:50.332817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-88be262f-836c-4728-b39e-5d7a4942cabe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:50.334365 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1403/2836] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:55:50 2022] DELETE /identity/v3/projects/9a7a3e7442784c17ac6136bd497e4680 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:50.339408 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f003561-4ef3-4a0d-8640-7720a9db010d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:50.339893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f003561-4ef3-4a0d-8640-7720a9db010d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:50.340031 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f003561-4ef3-4a0d-8640-7720a9db010d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:50.359391 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4f003561-4ef3-4a0d-8640-7720a9db010d None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:50.360664 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4f003561-4ef3-4a0d-8640-7720a9db010d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:50.422355 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1434/2837] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:50.431074 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebf2b9-04da-4c3c-a7b5-2f683295ec8c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:50.431367 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebf2b9-04da-4c3c-a7b5-2f683295ec8c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:50.431651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebf2b9-04da-4c3c-a7b5-2f683295ec8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:50.450405 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2cebf2b9-04da-4c3c-a7b5-2f683295ec8c None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:50.452282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2cebf2b9-04da-4c3c-a7b5-2f683295ec8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:50.460368 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1404/2838] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:50.464789 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f003561-4ef3-4a0d-8640-7720a9db010d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:50.475197 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f003561-4ef3-4a0d-8640-7720a9db010d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:50.479790 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863ea01-9456-49cf-ba8d-5fb9f9b6b7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:50.480472 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863ea01-9456-49cf-ba8d-5fb9f9b6b7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:50.480757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863ea01-9456-49cf-ba8d-5fb9f9b6b7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:50.481033 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863ea01-9456-49cf-ba8d-5fb9f9b6b7a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:50.481766 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863ea01-9456-49cf-ba8d-5fb9f9b6b7a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:55:50.482070 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863ea01-9456-49cf-ba8d-5fb9f9b6b7a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:50.484246 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863ea01-9456-49cf-ba8d-5fb9f9b6b7a9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:50.496725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1435/2839] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:55:50.502872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebf2b9-04da-4c3c-a7b5-2f683295ec8c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:50.512584 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebf2b9-04da-4c3c-a7b5-2f683295ec8c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:50.518404 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6b3c1-7386-49be-bf9e-e74bb4b8c527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:50.519064 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6b3c1-7386-49be-bf9e-e74bb4b8c527 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:50.519351 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6b3c1-7386-49be-bf9e-e74bb4b8c527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:50.519631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6b3c1-7386-49be-bf9e-e74bb4b8c527 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:50.520369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6b3c1-7386-49be-bf9e-e74bb4b8c527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:50.522292 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6b3c1-7386-49be-bf9e-e74bb4b8c527 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:50.549132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1405/2840] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:55:50 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:55:50.553489 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863ea01-9456-49cf-ba8d-5fb9f9b6b7a9 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:50.565495 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863ea01-9456-49cf-ba8d-5fb9f9b6b7a9 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:50.594223 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:50.627228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85eedb2-c136-4482-81cf-7513b57103c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:50.628092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85eedb2-c136-4482-81cf-7513b57103c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:50.628352 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85eedb2-c136-4482-81cf-7513b57103c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:50.628632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85eedb2-c136-4482-81cf-7513b57103c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:50.629360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85eedb2-c136-4482-81cf-7513b57103c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:50.631318 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85eedb2-c136-4482-81cf-7513b57103c0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:50.652926 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1436/2841] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:55:50 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:55:50.666079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6b3c1-7386-49be-bf9e-e74bb4b8c527 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:50.678166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6b3c1-7386-49be-bf9e-e74bb4b8c527 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:50.682759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab3a72-3eee-4f8b-85df-a331cb1ee155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:50.683429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab3a72-3eee-4f8b-85df-a331cb1ee155 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:50.683715 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab3a72-3eee-4f8b-85df-a331cb1ee155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:50.684023 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab3a72-3eee-4f8b-85df-a331cb1ee155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:50.684808 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab3a72-3eee-4f8b-85df-a331cb1ee155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:50.687771 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab3a72-3eee-4f8b-85df-a331cb1ee155 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:50.699313 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1406/2842] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:55:50.704107 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85eedb2-c136-4482-81cf-7513b57103c0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:50.715188 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85eedb2-c136-4482-81cf-7513b57103c0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:50.720406 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de14529-58fb-47ec-a70f-44b271d12132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:50.721130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de14529-58fb-47ec-a70f-44b271d12132 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:50.721454 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de14529-58fb-47ec-a70f-44b271d12132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:50.721769 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de14529-58fb-47ec-a70f-44b271d12132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:50.722718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de14529-58fb-47ec-a70f-44b271d12132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:50.724430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de14529-58fb-47ec-a70f-44b271d12132 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:50.736350 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1437/2843] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:55:50.746760 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab3a72-3eee-4f8b-85df-a331cb1ee155 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:50.751655 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab3a72-3eee-4f8b-85df-a331cb1ee155 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:50.756421 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a288bce8-28a6-4876-a0d3-5d053dca6e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:50.757323 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288bce8-28a6-4876-a0d3-5d053dca6e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:50.757603 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288bce8-28a6-4876-a0d3-5d053dca6e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:50.757893 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288bce8-28a6-4876-a0d3-5d053dca6e43 admin admin] PATH_INFO: `/v3/projects/7763c98c14ca45cb8e0d221ac55bba33/users/a74af6bca76c4dff9c216ad2eb57a430/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:50.773018 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a288bce8-28a6-4876-a0d3-5d053dca6e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7763c98c14ca45cb8e0d221ac55bba33, user_id=a74af6bca76c4dff9c216ad2eb57a430, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:50.775630 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a288bce8-28a6-4876-a0d3-5d053dca6e43 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:50.788872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1407/2844] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:55:50 2022] PUT /identity/v3/projects/7763c98c14ca45cb8e0d221ac55bba33/users/a74af6bca76c4dff9c216ad2eb57a430/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:50.800333 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46dfc17-d166-4307-9fbd-4a44739e9fc3 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:50.800633 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46dfc17-d166-4307-9fbd-4a44739e9fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:50.800918 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46dfc17-d166-4307-9fbd-4a44739e9fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:50.828720 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b46dfc17-d166-4307-9fbd-4a44739e9fc3 None None] MFA Rules not processed for user `a74af6bca76c4dff9c216ad2eb57a430`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:50.829955 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b46dfc17-d166-4307-9fbd-4a44739e9fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:50.896005 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1438/2845] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:55:50.901089 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a288bce8-28a6-4876-a0d3-5d053dca6e43 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:50.968968 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a288bce8-28a6-4876-a0d3-5d053dca6e43 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:50.973710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0ab94-c8af-4470-8f9d-178e9841ce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:50.974366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0ab94-c8af-4470-8f9d-178e9841ce7d admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:50.974650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0ab94-c8af-4470-8f9d-178e9841ce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:50.974956 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0ab94-c8af-4470-8f9d-178e9841ce7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:50.975650 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0ab94-c8af-4470-8f9d-178e9841ce7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:50.977615 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0ab94-c8af-4470-8f9d-178e9841ce7d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:51.010518 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1408/2846] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:55:50 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:55:51.015066 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46dfc17-d166-4307-9fbd-4a44739e9fc3 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:51.025974 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46dfc17-d166-4307-9fbd-4a44739e9fc3 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:51.087883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2728b-a0c1-4025-a342-f7be726bb82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:51.088605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2728b-a0c1-4025-a342-f7be726bb82a admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:51.088892 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2728b-a0c1-4025-a342-f7be726bb82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:51.089171 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2728b-a0c1-4025-a342-f7be726bb82a admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:51.089879 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2728b-a0c1-4025-a342-f7be726bb82a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:51.092779 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2728b-a0c1-4025-a342-f7be726bb82a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:51.114982 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1439/2847] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:55:51 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:55:51.119382 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0ab94-c8af-4470-8f9d-178e9841ce7d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:51.131372 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0ab94-c8af-4470-8f9d-178e9841ce7d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:51.136130 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdadae-58cd-42bf-824e-fdba6d983f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:51.136801 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdadae-58cd-42bf-824e-fdba6d983f2e admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:51.137091 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdadae-58cd-42bf-824e-fdba6d983f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:51.137372 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdadae-58cd-42bf-824e-fdba6d983f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:51.138101 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcdadae-58cd-42bf-824e-fdba6d983f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:51.140483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcdadae-58cd-42bf-824e-fdba6d983f2e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:51.151482 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1409/2848] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:55:51.155773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2728b-a0c1-4025-a342-f7be726bb82a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:51.166091 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2728b-a0c1-4025-a342-f7be726bb82a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:51.171448 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43812c-3890-411c-8d42-f1b3c2bc6e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:51.172376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43812c-3890-411c-8d42-f1b3c2bc6e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:51.172667 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43812c-3890-411c-8d42-f1b3c2bc6e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:51.172946 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43812c-3890-411c-8d42-f1b3c2bc6e1a admin admin] PATH_INFO: `/v3/projects/fcb7114de71649f7bff969d7eb8b98e5/users/626eea3e0fc34d5bbfb8a08ae0e25dac/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:51.189125 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43812c-3890-411c-8d42-f1b3c2bc6e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb7114de71649f7bff969d7eb8b98e5, user_id=626eea3e0fc34d5bbfb8a08ae0e25dac, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:51.191858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43812c-3890-411c-8d42-f1b3c2bc6e1a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:51.205958 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1440/2849] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:55:51 2022] PUT /identity/v3/projects/fcb7114de71649f7bff969d7eb8b98e5/users/626eea3e0fc34d5bbfb8a08ae0e25dac/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:51.209963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdadae-58cd-42bf-824e-fdba6d983f2e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:51.220191 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdadae-58cd-42bf-824e-fdba6d983f2e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:51.240229 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:51.287524 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534ec457-abd4-4e36-945f-8be972998ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:51.288186 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ec457-abd4-4e36-945f-8be972998ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:51.288468 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ec457-abd4-4e36-945f-8be972998ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:51.288739 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ec457-abd4-4e36-945f-8be972998ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:51.289475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534ec457-abd4-4e36-945f-8be972998ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:51.291104 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534ec457-abd4-4e36-945f-8be972998ec5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:51.301920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1410/2850] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:55:51.306159 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43812c-3890-411c-8d42-f1b3c2bc6e1a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:51.312297 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:51.319160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43812c-3890-411c-8d42-f1b3c2bc6e1a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:51.324100 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1d7b2-5267-4551-bea2-94702602fbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:51.324991 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1d7b2-5267-4551-bea2-94702602fbfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:51.325277 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1d7b2-5267-4551-bea2-94702602fbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:51.325555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1d7b2-5267-4551-bea2-94702602fbfa admin admin] PATH_INFO: `/v3/projects/fcb7114de71649f7bff969d7eb8b98e5/users/626eea3e0fc34d5bbfb8a08ae0e25dac/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:51.328245 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df1d7b2-5267-4551-bea2-94702602fbfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb7114de71649f7bff969d7eb8b98e5, user_id=626eea3e0fc34d5bbfb8a08ae0e25dac, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:51.330923 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df1d7b2-5267-4551-bea2-94702602fbfa admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:51.341646 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1441/2851] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:55:51 2022] PUT /identity/v3/projects/fcb7114de71649f7bff969d7eb8b98e5/users/626eea3e0fc34d5bbfb8a08ae0e25dac/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:55:51.352024 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e57962-8acb-4368-b579-fe0a0fe06068 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:51.352319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e57962-8acb-4368-b579-fe0a0fe06068 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:51.352602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e57962-8acb-4368-b579-fe0a0fe06068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:51.380376 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-23e57962-8acb-4368-b579-fe0a0fe06068 None None] MFA Rules not processed for user `626eea3e0fc34d5bbfb8a08ae0e25dac`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:51.381599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-23e57962-8acb-4368-b579-fe0a0fe06068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:51.450194 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1411/2852] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:51.461555 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1d7b2-5267-4551-bea2-94702602fbfa admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:51.463293 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0df1d7b2-5267-4551-bea2-94702602fbfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:51.521165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df1d7b2-5267-4551-bea2-94702602fbfa admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:51.524840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cf259-3410-4202-929e-6e884832b730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:51.525513 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cf259-3410-4202-929e-6e884832b730 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:51.525797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cf259-3410-4202-929e-6e884832b730 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:51.526075 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cf259-3410-4202-929e-6e884832b730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:51.535582 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cf259-3410-4202-929e-6e884832b730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:51.537082 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cf259-3410-4202-929e-6e884832b730 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:51.547675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1442/2853] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:55:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:55:51.613286 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e57962-8acb-4368-b579-fe0a0fe06068 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:51.677006 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e57962-8acb-4368-b579-fe0a0fe06068 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:51.681440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb32bc3-ffc8-413c-b4a2-d27fbc474824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:51.682296 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb32bc3-ffc8-413c-b4a2-d27fbc474824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:51.682577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb32bc3-ffc8-413c-b4a2-d27fbc474824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:51.682850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb32bc3-ffc8-413c-b4a2-d27fbc474824 admin admin] PATH_INFO: `/v3/users/a74af6bca76c4dff9c216ad2eb57a430` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:51.684116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb32bc3-ffc8-413c-b4a2-d27fbc474824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a74af6bca76c4dff9c216ad2eb57a430)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:51.686674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb32bc3-ffc8-413c-b4a2-d27fbc474824 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:51.741521 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1fb32bc3-ffc8-413c-b4a2-d27fbc474824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a74af6bca76c4dff9c216ad2eb57a430'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:51.743371 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:51.751765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1fb32bc3-ffc8-413c-b4a2-d27fbc474824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a74af6bca76c4dff9c216ad2eb57a430'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:51.760322 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1fb32bc3-ffc8-413c-b4a2-d27fbc474824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a74af6bca76c4dff9c216ad2eb57a430'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:51.773870 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1fb32bc3-ffc8-413c-b4a2-d27fbc474824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a74af6bca76c4dff9c216ad2eb57a430'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:51.774606 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1fb32bc3-ffc8-413c-b4a2-d27fbc474824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a74af6bca76c4dff9c216ad2eb57a430'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:51.781658 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1412/2854] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:55:51 2022] DELETE /identity/v3/users/a74af6bca76c4dff9c216ad2eb57a430 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:51.786241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737494e-9039-43fa-bf62-d9fdf67fd5a1 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:51.786549 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737494e-9039-43fa-bf62-d9fdf67fd5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:51.786836 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737494e-9039-43fa-bf62-d9fdf67fd5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:51.805301 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f737494e-9039-43fa-bf62-d9fdf67fd5a1 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:51.806423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f737494e-9039-43fa-bf62-d9fdf67fd5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:51.864014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1443/2855] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:55:51.873901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb32bc3-ffc8-413c-b4a2-d27fbc474824 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:51.875566 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1fb32bc3-ffc8-413c-b4a2-d27fbc474824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:51.933467 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb32bc3-ffc8-413c-b4a2-d27fbc474824 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:51.937345 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c84b0-8559-4962-a855-f6d55fbe9193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:51.937979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c84b0-8559-4962-a855-f6d55fbe9193 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:51.938258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c84b0-8559-4962-a855-f6d55fbe9193 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:51.938532 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c84b0-8559-4962-a855-f6d55fbe9193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:51.947577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c84b0-8559-4962-a855-f6d55fbe9193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:51.949054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c84b0-8559-4962-a855-f6d55fbe9193 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:51.958651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1413/2856] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:55:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:55:52.233241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737494e-9039-43fa-bf62-d9fdf67fd5a1 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:52.234947 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f737494e-9039-43fa-bf62-d9fdf67fd5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:52.248998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737494e-9039-43fa-bf62-d9fdf67fd5a1 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:52.253638 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fcebe-460f-49d0-85aa-bcb910da468b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:52.255622 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fcebe-460f-49d0-85aa-bcb910da468b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:52.255788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fcebe-460f-49d0-85aa-bcb910da468b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:52.256108 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fcebe-460f-49d0-85aa-bcb910da468b admin admin] PATH_INFO: `/v3/projects/7763c98c14ca45cb8e0d221ac55bba33` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:52.257336 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fcebe-460f-49d0-85aa-bcb910da468b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7763c98c14ca45cb8e0d221ac55bba33)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:52.259172 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fcebe-460f-49d0-85aa-bcb910da468b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:52.320666 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b56fcebe-460f-49d0-85aa-bcb910da468b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7763c98c14ca45cb8e0d221ac55bba33', 'request_id': 'req-b56fcebe-460f-49d0-85aa-bcb910da468b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:52.326721 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b56fcebe-460f-49d0-85aa-bcb910da468b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7763c98c14ca45cb8e0d221ac55bba33', 'request_id': 'req-b56fcebe-460f-49d0-85aa-bcb910da468b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:52.333591 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b56fcebe-460f-49d0-85aa-bcb910da468b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7763c98c14ca45cb8e0d221ac55bba33', 'request_id': 'req-b56fcebe-460f-49d0-85aa-bcb910da468b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:52.347510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b56fcebe-460f-49d0-85aa-bcb910da468b admin admin] The token cache is being invalidate because project 7763c98c14ca45cb8e0d221ac55bba33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:55:52.348513 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b56fcebe-460f-49d0-85aa-bcb910da468b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:52.349165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b56fcebe-460f-49d0-85aa-bcb910da468b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:52.350771 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1444/2857] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:55:52 2022] DELETE /identity/v3/projects/7763c98c14ca45cb8e0d221ac55bba33 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:52.354697 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c84b0-8559-4962-a855-f6d55fbe9193 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:52.364881 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c84b0-8559-4962-a855-f6d55fbe9193 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:52.423900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a87585-1814-4e03-a451-4d731485a939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:52.424806 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a87585-1814-4e03-a451-4d731485a939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:52.425092 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a87585-1814-4e03-a451-4d731485a939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:52.425359 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a87585-1814-4e03-a451-4d731485a939 admin admin] PATH_INFO: `/v3/users/626eea3e0fc34d5bbfb8a08ae0e25dac` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:52.426585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a87585-1814-4e03-a451-4d731485a939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=626eea3e0fc34d5bbfb8a08ae0e25dac)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:52.428553 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a87585-1814-4e03-a451-4d731485a939 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:52.474768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-76a87585-1814-4e03-a451-4d731485a939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '626eea3e0fc34d5bbfb8a08ae0e25dac'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:52.482073 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-76a87585-1814-4e03-a451-4d731485a939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '626eea3e0fc34d5bbfb8a08ae0e25dac'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:52.489145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-76a87585-1814-4e03-a451-4d731485a939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '626eea3e0fc34d5bbfb8a08ae0e25dac'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:52.494187 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:52.503056 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-76a87585-1814-4e03-a451-4d731485a939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '626eea3e0fc34d5bbfb8a08ae0e25dac'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:52.503987 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-76a87585-1814-4e03-a451-4d731485a939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '626eea3e0fc34d5bbfb8a08ae0e25dac'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:52.510162 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1414/2858] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:55:52 2022] DELETE /identity/v3/users/626eea3e0fc34d5bbfb8a08ae0e25dac => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:52.797061 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fcebe-460f-49d0-85aa-bcb910da468b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:52.798575 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b56fcebe-460f-49d0-85aa-bcb910da468b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:52.813303 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fcebe-460f-49d0-85aa-bcb910da468b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:52.825765 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:52.874801 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625a33a-f10c-460f-bd03-4226865bf149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:52.875596 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625a33a-f10c-460f-bd03-4226865bf149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:52.875883 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625a33a-f10c-460f-bd03-4226865bf149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:52.876190 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625a33a-f10c-460f-bd03-4226865bf149 admin admin] PATH_INFO: `/v3/projects/fcb7114de71649f7bff969d7eb8b98e5` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:52.877416 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625a33a-f10c-460f-bd03-4226865bf149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcb7114de71649f7bff969d7eb8b98e5)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:52.879333 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625a33a-f10c-460f-bd03-4226865bf149 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:52.927138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c625a33a-f10c-460f-bd03-4226865bf149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcb7114de71649f7bff969d7eb8b98e5', 'request_id': 'req-c625a33a-f10c-460f-bd03-4226865bf149'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:52.932621 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c625a33a-f10c-460f-bd03-4226865bf149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcb7114de71649f7bff969d7eb8b98e5', 'request_id': 'req-c625a33a-f10c-460f-bd03-4226865bf149'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:52.939804 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c625a33a-f10c-460f-bd03-4226865bf149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcb7114de71649f7bff969d7eb8b98e5', 'request_id': 'req-c625a33a-f10c-460f-bd03-4226865bf149'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:52.953387 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c625a33a-f10c-460f-bd03-4226865bf149 admin admin] The token cache is being invalidate because project fcb7114de71649f7bff969d7eb8b98e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:55:52.953675 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c625a33a-f10c-460f-bd03-4226865bf149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:52.954294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c625a33a-f10c-460f-bd03-4226865bf149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:52.955902 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1445/2859] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:55:52 2022] DELETE /identity/v3/projects/fcb7114de71649f7bff969d7eb8b98e5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:52.960931 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17e3c2-612c-4031-97b6-a18834688516 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:52.961233 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17e3c2-612c-4031-97b6-a18834688516 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:52.961516 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d17e3c2-612c-4031-97b6-a18834688516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:52.980265 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5d17e3c2-612c-4031-97b6-a18834688516 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:52.981434 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5d17e3c2-612c-4031-97b6-a18834688516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:53.042336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1415/2860] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:53.052404 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca9abb-5750-4afd-8065-508add95ac05 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:53.052655 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca9abb-5750-4afd-8065-508add95ac05 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:53.052941 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca9abb-5750-4afd-8065-508add95ac05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:53.071800 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-03ca9abb-5750-4afd-8065-508add95ac05 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:53.073078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-03ca9abb-5750-4afd-8065-508add95ac05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:53.083387 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1446/2861] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:53.087877 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17e3c2-612c-4031-97b6-a18834688516 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:53.097828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d17e3c2-612c-4031-97b6-a18834688516 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:53.102459 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e321d-fc5d-4c0c-91c5-36b4438b32d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:53.103110 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e321d-fc5d-4c0c-91c5-36b4438b32d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:53.103397 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e321d-fc5d-4c0c-91c5-36b4438b32d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:53.103711 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e321d-fc5d-4c0c-91c5-36b4438b32d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:53.104485 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e321d-fc5d-4c0c-91c5-36b4438b32d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:55:53.104796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e321d-fc5d-4c0c-91c5-36b4438b32d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:53.106415 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e321d-fc5d-4c0c-91c5-36b4438b32d4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:53.118602 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1416/2862] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:55:53.123449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca9abb-5750-4afd-8065-508add95ac05 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:53.132880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca9abb-5750-4afd-8065-508add95ac05 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:53.137512 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0eabe-5678-4a4c-b622-f3baf84696c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:53.138152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0eabe-5678-4a4c-b622-f3baf84696c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:53.139357 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0eabe-5678-4a4c-b622-f3baf84696c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:53.139641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0eabe-5678-4a4c-b622-f3baf84696c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:53.140373 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0eabe-5678-4a4c-b622-f3baf84696c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:53.142267 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0eabe-5678-4a4c-b622-f3baf84696c3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:53.168716 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1447/2863] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:55:53 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:55:53.172855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e321d-fc5d-4c0c-91c5-36b4438b32d4 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:53.182940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e321d-fc5d-4c0c-91c5-36b4438b32d4 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:53.238751 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47119635-2d29-46a9-b82e-91e7cb5d685a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:53.239409 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47119635-2d29-46a9-b82e-91e7cb5d685a admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:53.239703 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47119635-2d29-46a9-b82e-91e7cb5d685a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:53.240008 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47119635-2d29-46a9-b82e-91e7cb5d685a admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:53.240696 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47119635-2d29-46a9-b82e-91e7cb5d685a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:53.242634 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47119635-2d29-46a9-b82e-91e7cb5d685a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:53.263197 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1417/2864] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:55:53 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:55:53.268431 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0eabe-5678-4a4c-b622-f3baf84696c3 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:53.278653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0eabe-5678-4a4c-b622-f3baf84696c3 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:53.284073 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f7c90-9c33-4be2-8192-64133d3b4973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:53.284730 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f7c90-9c33-4be2-8192-64133d3b4973 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:53.285012 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f7c90-9c33-4be2-8192-64133d3b4973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:53.285368 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f7c90-9c33-4be2-8192-64133d3b4973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:53.286010 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f7c90-9c33-4be2-8192-64133d3b4973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:53.287608 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f7c90-9c33-4be2-8192-64133d3b4973 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:53.299018 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1448/2865] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:55:53.302960 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47119635-2d29-46a9-b82e-91e7cb5d685a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:53.313143 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47119635-2d29-46a9-b82e-91e7cb5d685a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:53.317817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd085a-4c18-4de7-b527-e77002c17061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:53.318474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd085a-4c18-4de7-b527-e77002c17061 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:53.318759 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd085a-4c18-4de7-b527-e77002c17061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:53.319033 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd085a-4c18-4de7-b527-e77002c17061 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:53.319749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd085a-4c18-4de7-b527-e77002c17061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:53.321396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd085a-4c18-4de7-b527-e77002c17061 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:53.334487 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1418/2866] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:55:53.337409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f7c90-9c33-4be2-8192-64133d3b4973 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:53.348178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f7c90-9c33-4be2-8192-64133d3b4973 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:53.352803 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47b136-f92f-487c-879b-149cd43227f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:53.353680 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47b136-f92f-487c-879b-149cd43227f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:53.353963 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47b136-f92f-487c-879b-149cd43227f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:53.354246 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47b136-f92f-487c-879b-149cd43227f9 admin admin] PATH_INFO: `/v3/projects/19f3f0c2cf334768abcee42023d95427/users/e8f460289b8a4c19aab96fb0c45d0d62/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:53.370021 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47b136-f92f-487c-879b-149cd43227f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f3f0c2cf334768abcee42023d95427, user_id=e8f460289b8a4c19aab96fb0c45d0d62, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:53.374031 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47b136-f92f-487c-879b-149cd43227f9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:53.386916 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1449/2867] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:55:53 2022] PUT /identity/v3/projects/19f3f0c2cf334768abcee42023d95427/users/e8f460289b8a4c19aab96fb0c45d0d62/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:55:53.397829 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01072b50-0799-4efe-8f41-983d149cd6ed None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:53.398316 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01072b50-0799-4efe-8f41-983d149cd6ed None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:53.398407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01072b50-0799-4efe-8f41-983d149cd6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:53.426457 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-01072b50-0799-4efe-8f41-983d149cd6ed None None] MFA Rules not processed for user `e8f460289b8a4c19aab96fb0c45d0d62`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:53.427695 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-01072b50-0799-4efe-8f41-983d149cd6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:53.468646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:53.493782 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1419/2868] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:53.498471 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47b136-f92f-487c-879b-149cd43227f9 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:53.544597 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:53.563519 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47b136-f92f-487c-879b-149cd43227f9 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:53.568454 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5e925-331e-4cba-acf9-3706c4b30026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:53.569111 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5e925-331e-4cba-acf9-3706c4b30026 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:53.569393 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5e925-331e-4cba-acf9-3706c4b30026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:53.569697 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5e925-331e-4cba-acf9-3706c4b30026 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:53.570425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5e925-331e-4cba-acf9-3706c4b30026 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:53.572411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5e925-331e-4cba-acf9-3706c4b30026 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:53.600916 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1450/2869] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:55:53 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:55:53.602063 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01072b50-0799-4efe-8f41-983d149cd6ed None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:53.613865 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01072b50-0799-4efe-8f41-983d149cd6ed None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:53.671319 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9ee52-7aeb-40bf-9143-d47630684476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:53.671897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9ee52-7aeb-40bf-9143-d47630684476 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:53.672210 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9ee52-7aeb-40bf-9143-d47630684476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:53.672486 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9ee52-7aeb-40bf-9143-d47630684476 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:53.673185 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9ee52-7aeb-40bf-9143-d47630684476 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:53.675137 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9ee52-7aeb-40bf-9143-d47630684476 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:53.695392 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1420/2870] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:55:53 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:55:53.699283 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5e925-331e-4cba-acf9-3706c4b30026 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:53.710225 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5e925-331e-4cba-acf9-3706c4b30026 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:53.715019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e2b0d-054d-4582-ba5c-55197b34cd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:53.715657 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e2b0d-054d-4582-ba5c-55197b34cd62 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:53.715970 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e2b0d-054d-4582-ba5c-55197b34cd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:53.716248 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e2b0d-054d-4582-ba5c-55197b34cd62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:53.717788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e2b0d-054d-4582-ba5c-55197b34cd62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:53.719524 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e2b0d-054d-4582-ba5c-55197b34cd62 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:53.730509 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1451/2871] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:55:53.734716 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9ee52-7aeb-40bf-9143-d47630684476 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:53.746121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9ee52-7aeb-40bf-9143-d47630684476 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:53.751640 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df40e8-e8fc-40c6-b019-a8c5b1a7acdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:53.752732 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df40e8-e8fc-40c6-b019-a8c5b1a7acdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:53.753016 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df40e8-e8fc-40c6-b019-a8c5b1a7acdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:53.753294 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df40e8-e8fc-40c6-b019-a8c5b1a7acdd admin admin] PATH_INFO: `/v3/projects/0a6f9657c7d54b40afef1022083baa33/users/88b4409cbf014a73b5b2c6bac53931cb/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:53.769710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df40e8-e8fc-40c6-b019-a8c5b1a7acdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a6f9657c7d54b40afef1022083baa33, user_id=88b4409cbf014a73b5b2c6bac53931cb, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:53.772418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df40e8-e8fc-40c6-b019-a8c5b1a7acdd admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:53.785797 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1421/2872] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:55:53 2022] PUT /identity/v3/projects/0a6f9657c7d54b40afef1022083baa33/users/88b4409cbf014a73b5b2c6bac53931cb/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:53.789811 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e2b0d-054d-4582-ba5c-55197b34cd62 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:53.800468 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e2b0d-054d-4582-ba5c-55197b34cd62 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:53.857884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96eab26-a64c-46fd-a869-88fc8fe73bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:53.858545 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eab26-a64c-46fd-a869-88fc8fe73bbd admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:53.858834 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eab26-a64c-46fd-a869-88fc8fe73bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:53.859113 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eab26-a64c-46fd-a869-88fc8fe73bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:53.859842 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96eab26-a64c-46fd-a869-88fc8fe73bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:53.861509 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96eab26-a64c-46fd-a869-88fc8fe73bbd admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:53.871820 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1452/2873] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:55:53.875872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df40e8-e8fc-40c6-b019-a8c5b1a7acdd admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:53.886519 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df40e8-e8fc-40c6-b019-a8c5b1a7acdd admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:53.892071 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d6718-4137-4e3c-858a-1be8bc138710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:53.892954 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d6718-4137-4e3c-858a-1be8bc138710 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:53.893239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d6718-4137-4e3c-858a-1be8bc138710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:53.893520 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d6718-4137-4e3c-858a-1be8bc138710 admin admin] PATH_INFO: `/v3/projects/0a6f9657c7d54b40afef1022083baa33/users/88b4409cbf014a73b5b2c6bac53931cb/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:53.895541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d6718-4137-4e3c-858a-1be8bc138710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a6f9657c7d54b40afef1022083baa33, user_id=88b4409cbf014a73b5b2c6bac53931cb, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:53.898245 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d6718-4137-4e3c-858a-1be8bc138710 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:53.909250 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1422/2874] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:55:53 2022] PUT /identity/v3/projects/0a6f9657c7d54b40afef1022083baa33/users/88b4409cbf014a73b5b2c6bac53931cb/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:53.920207 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ded547-18be-49b1-ab57-27fb65a610cc None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:53.920505 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ded547-18be-49b1-ab57-27fb65a610cc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:53.920787 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ded547-18be-49b1-ab57-27fb65a610cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:53.947336 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-06ded547-18be-49b1-ab57-27fb65a610cc None None] MFA Rules not processed for user `88b4409cbf014a73b5b2c6bac53931cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:53.948493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-06ded547-18be-49b1-ab57-27fb65a610cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:54.017762 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1453/2875] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:54.031935 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d6718-4137-4e3c-858a-1be8bc138710 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:54.033654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7a3d6718-4137-4e3c-858a-1be8bc138710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:54.093692 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d6718-4137-4e3c-858a-1be8bc138710 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:54.097409 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a599ce0-e53f-4573-bcef-ea68391c1e1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:54.098050 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a599ce0-e53f-4573-bcef-ea68391c1e1f service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:54.098336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a599ce0-e53f-4573-bcef-ea68391c1e1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:54.098616 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a599ce0-e53f-4573-bcef-ea68391c1e1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:54.107776 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a599ce0-e53f-4573-bcef-ea68391c1e1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:54.109971 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a599ce0-e53f-4573-bcef-ea68391c1e1f service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:54.119396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1423/2876] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:55:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:55:54.431092 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ded547-18be-49b1-ab57-27fb65a610cc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:54.440573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ded547-18be-49b1-ab57-27fb65a610cc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:54.445160 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb2f72-3b4e-4b37-97db-daefdba01684 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b4409cbf014a73b5b2c6bac53931cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a6f9657c7d54b40afef1022083baa33', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:54.445812 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb2f72-3b4e-4b37-97db-daefdba01684 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:54.446104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb2f72-3b4e-4b37-97db-daefdba01684 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:54.446389 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb2f72-3b4e-4b37-97db-daefdba01684 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:54.447119 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb2f72-3b4e-4b37-97db-daefdba01684 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:55:54.447423 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb2f72-3b4e-4b37-97db-daefdba01684 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:54.449053 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb2f72-3b4e-4b37-97db-daefdba01684 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:54.461315 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1454/2877] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:55:54.465357 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a599ce0-e53f-4573-bcef-ea68391c1e1f service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:54.475260 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a599ce0-e53f-4573-bcef-ea68391c1e1f service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:54.479898 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae214bf-30c4-4903-a1d3-a0aa11990374 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b4409cbf014a73b5b2c6bac53931cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a6f9657c7d54b40afef1022083baa33', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:54.480573 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae214bf-30c4-4903-a1d3-a0aa11990374 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:54.480867 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae214bf-30c4-4903-a1d3-a0aa11990374 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:54.481147 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae214bf-30c4-4903-a1d3-a0aa11990374 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:54.481834 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae214bf-30c4-4903-a1d3-a0aa11990374 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:54.483846 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae214bf-30c4-4903-a1d3-a0aa11990374 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:54.509048 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1424/2878] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:55:54 2022] POST /identity/v3/projects => generated 353 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:55:54.513055 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb2f72-3b4e-4b37-97db-daefdba01684 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:54.524296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb2f72-3b4e-4b37-97db-daefdba01684 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:54.546296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:54.583943 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ad219-fb2e-4cc9-b87e-9635f526038e tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b4409cbf014a73b5b2c6bac53931cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a6f9657c7d54b40afef1022083baa33', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:54.584625 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ad219-fb2e-4cc9-b87e-9635f526038e tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:54.584909 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ad219-fb2e-4cc9-b87e-9635f526038e tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:54.585186 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ad219-fb2e-4cc9-b87e-9635f526038e tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:54.585907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ad219-fb2e-4cc9-b87e-9635f526038e tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:55:54.586214 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ad219-fb2e-4cc9-b87e-9635f526038e tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:54.587810 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ad219-fb2e-4cc9-b87e-9635f526038e tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:54.601571 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1455/2879] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:55:54.641586 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae214bf-30c4-4903-a1d3-a0aa11990374 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:54.652563 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae214bf-30c4-4903-a1d3-a0aa11990374 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:54.659624 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789e1e3-6236-4556-b088-abfae1a95e60 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88b4409cbf014a73b5b2c6bac53931cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0a6f9657c7d54b40afef1022083baa33', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:54.660977 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789e1e3-6236-4556-b088-abfae1a95e60 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:54.661342 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789e1e3-6236-4556-b088-abfae1a95e60 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:54.661717 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789e1e3-6236-4556-b088-abfae1a95e60 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] PATH_INFO: `/v3/projects/4c24100cdb58476aa9584f6aa983ee6b` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:54.663320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789e1e3-6236-4556-b088-abfae1a95e60 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4c24100cdb58476aa9584f6aa983ee6b)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:54.666067 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789e1e3-6236-4556-b088-abfae1a95e60 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:54.683187 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:54.729910 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2789e1e3-6236-4556-b088-abfae1a95e60 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c24100cdb58476aa9584f6aa983ee6b', 'request_id': 'req-2789e1e3-6236-4556-b088-abfae1a95e60'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:54.736828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2789e1e3-6236-4556-b088-abfae1a95e60 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c24100cdb58476aa9584f6aa983ee6b', 'request_id': 'req-2789e1e3-6236-4556-b088-abfae1a95e60'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:54.745235 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2789e1e3-6236-4556-b088-abfae1a95e60 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c24100cdb58476aa9584f6aa983ee6b', 'request_id': 'req-2789e1e3-6236-4556-b088-abfae1a95e60'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:54.759874 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2789e1e3-6236-4556-b088-abfae1a95e60 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] The token cache is being invalidate because project 4c24100cdb58476aa9584f6aa983ee6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:55:54.760198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2789e1e3-6236-4556-b088-abfae1a95e60 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:54.760927 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2789e1e3-6236-4556-b088-abfae1a95e60 tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:54.762548 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1425/2880] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:55:54 2022] DELETE /identity/v3/projects/4c24100cdb58476aa9584f6aa983ee6b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:54.974376 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ad219-fb2e-4cc9-b87e-9635f526038e tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:54.976142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-211ad219-fb2e-4cc9-b87e-9635f526038e tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:55.049104 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ad219-fb2e-4cc9-b87e-9635f526038e tempest-VolumeQuotaClassesTest-1331068579 tempest-VolumeQuotaClassesTest-1331068579-project-admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:55.104756 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95208b0-7f08-4246-92e1-e778fe4548db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:55.105639 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95208b0-7f08-4246-92e1-e778fe4548db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:55.105927 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95208b0-7f08-4246-92e1-e778fe4548db admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:55.106206 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95208b0-7f08-4246-92e1-e778fe4548db admin admin] PATH_INFO: `/v3/users/e8f460289b8a4c19aab96fb0c45d0d62` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:55.107567 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95208b0-7f08-4246-92e1-e778fe4548db admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8f460289b8a4c19aab96fb0c45d0d62)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:55.110025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95208b0-7f08-4246-92e1-e778fe4548db admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:55.156296 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f95208b0-7f08-4246-92e1-e778fe4548db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8f460289b8a4c19aab96fb0c45d0d62'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:55.156937 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f95208b0-7f08-4246-92e1-e778fe4548db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8f460289b8a4c19aab96fb0c45d0d62'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:55.163939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f95208b0-7f08-4246-92e1-e778fe4548db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8f460289b8a4c19aab96fb0c45d0d62'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:55.174907 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:55.177654 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f95208b0-7f08-4246-92e1-e778fe4548db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8f460289b8a4c19aab96fb0c45d0d62'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:55.185819 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f95208b0-7f08-4246-92e1-e778fe4548db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8f460289b8a4c19aab96fb0c45d0d62'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:55.192406 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1456/2881] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:55:54 2022] DELETE /identity/v3/users/e8f460289b8a4c19aab96fb0c45d0d62 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:55.196963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb230c58-283e-40ec-9777-3c1f70710cf1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:55.197284 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb230c58-283e-40ec-9777-3c1f70710cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:55.197570 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb230c58-283e-40ec-9777-3c1f70710cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:55.215445 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-bb230c58-283e-40ec-9777-3c1f70710cf1 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:55.216740 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bb230c58-283e-40ec-9777-3c1f70710cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:55.277061 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1426/2882] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:55.291593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95208b0-7f08-4246-92e1-e778fe4548db admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:55.298135 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f95208b0-7f08-4246-92e1-e778fe4548db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:55.360632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95208b0-7f08-4246-92e1-e778fe4548db admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:55.365181 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27386d-0072-4835-a7b1-75f7967e184f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:55.365823 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27386d-0072-4835-a7b1-75f7967e184f service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:55.366103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27386d-0072-4835-a7b1-75f7967e184f service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:55.366379 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27386d-0072-4835-a7b1-75f7967e184f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:55.376289 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27386d-0072-4835-a7b1-75f7967e184f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:55.377733 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27386d-0072-4835-a7b1-75f7967e184f service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:55.387261 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1457/2883] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:55:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:55:55.658583 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb230c58-283e-40ec-9777-3c1f70710cf1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:55.660904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bb230c58-283e-40ec-9777-3c1f70710cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:55.675300 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb230c58-283e-40ec-9777-3c1f70710cf1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:55.680094 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83574dce-c150-4399-a4ef-aaaf89a88dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:55.680963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83574dce-c150-4399-a4ef-aaaf89a88dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:55.681240 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83574dce-c150-4399-a4ef-aaaf89a88dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:55.681515 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83574dce-c150-4399-a4ef-aaaf89a88dc8 admin admin] PATH_INFO: `/v3/projects/19f3f0c2cf334768abcee42023d95427` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:55.682811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83574dce-c150-4399-a4ef-aaaf89a88dc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f3f0c2cf334768abcee42023d95427)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:55.684853 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83574dce-c150-4399-a4ef-aaaf89a88dc8 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:55.732407 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-83574dce-c150-4399-a4ef-aaaf89a88dc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f3f0c2cf334768abcee42023d95427', 'request_id': 'req-83574dce-c150-4399-a4ef-aaaf89a88dc8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:55.739184 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-83574dce-c150-4399-a4ef-aaaf89a88dc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f3f0c2cf334768abcee42023d95427', 'request_id': 'req-83574dce-c150-4399-a4ef-aaaf89a88dc8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:55.753107 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-83574dce-c150-4399-a4ef-aaaf89a88dc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f3f0c2cf334768abcee42023d95427', 'request_id': 'req-83574dce-c150-4399-a4ef-aaaf89a88dc8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:55.765544 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-83574dce-c150-4399-a4ef-aaaf89a88dc8 admin admin] The token cache is being invalidate because project 19f3f0c2cf334768abcee42023d95427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:55:55.765973 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-83574dce-c150-4399-a4ef-aaaf89a88dc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:55.766787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-83574dce-c150-4399-a4ef-aaaf89a88dc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:55.769244 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1427/2884] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:55:55 2022] DELETE /identity/v3/projects/19f3f0c2cf334768abcee42023d95427 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:55.773486 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27386d-0072-4835-a7b1-75f7967e184f service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:55.783251 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27386d-0072-4835-a7b1-75f7967e184f service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:55.839801 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63bd45-956c-4b4f-8bbf-dd5e85bea251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:55.840700 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63bd45-956c-4b4f-8bbf-dd5e85bea251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:55.840988 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63bd45-956c-4b4f-8bbf-dd5e85bea251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:55.841269 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63bd45-956c-4b4f-8bbf-dd5e85bea251 admin admin] PATH_INFO: `/v3/users/88b4409cbf014a73b5b2c6bac53931cb` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:55.842449 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63bd45-956c-4b4f-8bbf-dd5e85bea251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88b4409cbf014a73b5b2c6bac53931cb)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:55.844986 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63bd45-956c-4b4f-8bbf-dd5e85bea251 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:55.891531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0e63bd45-956c-4b4f-8bbf-dd5e85bea251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88b4409cbf014a73b5b2c6bac53931cb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:55.892201 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0e63bd45-956c-4b4f-8bbf-dd5e85bea251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88b4409cbf014a73b5b2c6bac53931cb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:55.899865 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0e63bd45-956c-4b4f-8bbf-dd5e85bea251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88b4409cbf014a73b5b2c6bac53931cb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:55.911759 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0e63bd45-956c-4b4f-8bbf-dd5e85bea251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88b4409cbf014a73b5b2c6bac53931cb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:55.918725 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0e63bd45-956c-4b4f-8bbf-dd5e85bea251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88b4409cbf014a73b5b2c6bac53931cb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:55.925117 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1458/2885] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:55:55 2022] DELETE /identity/v3/users/88b4409cbf014a73b5b2c6bac53931cb => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 09:55:56.210396 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83574dce-c150-4399-a4ef-aaaf89a88dc8 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:56.211952 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-83574dce-c150-4399-a4ef-aaaf89a88dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:56.226151 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83574dce-c150-4399-a4ef-aaaf89a88dc8 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:56.245897 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:56.287734 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35105042-18df-4940-82df-d713a36e24a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:56.288629 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35105042-18df-4940-82df-d713a36e24a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:56.288917 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35105042-18df-4940-82df-d713a36e24a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:56.289198 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35105042-18df-4940-82df-d713a36e24a3 admin admin] PATH_INFO: `/v3/projects/0a6f9657c7d54b40afef1022083baa33` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:56.290389 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35105042-18df-4940-82df-d713a36e24a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a6f9657c7d54b40afef1022083baa33)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:56.292619 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35105042-18df-4940-82df-d713a36e24a3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:56.338803 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-35105042-18df-4940-82df-d713a36e24a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a6f9657c7d54b40afef1022083baa33', 'request_id': 'req-35105042-18df-4940-82df-d713a36e24a3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:56.345219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-35105042-18df-4940-82df-d713a36e24a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a6f9657c7d54b40afef1022083baa33', 'request_id': 'req-35105042-18df-4940-82df-d713a36e24a3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:56.353241 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-35105042-18df-4940-82df-d713a36e24a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a6f9657c7d54b40afef1022083baa33', 'request_id': 'req-35105042-18df-4940-82df-d713a36e24a3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:56.364095 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-35105042-18df-4940-82df-d713a36e24a3 admin admin] The token cache is being invalidate because project 0a6f9657c7d54b40afef1022083baa33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:55:56.364381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-35105042-18df-4940-82df-d713a36e24a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:56.365145 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-35105042-18df-4940-82df-d713a36e24a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:55:56.366711 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1428/2886] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:55:56 2022] DELETE /identity/v3/projects/0a6f9657c7d54b40afef1022083baa33 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:56.371709 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebc8ed-1295-4a10-aa99-eff38134de10 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:56.372151 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebc8ed-1295-4a10-aa99-eff38134de10 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:56.372527 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebc8ed-1295-4a10-aa99-eff38134de10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:56.386300 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:56.394323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-43ebc8ed-1295-4a10-aa99-eff38134de10 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:56.396444 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-43ebc8ed-1295-4a10-aa99-eff38134de10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:56.457454 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1459/2887] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:56.465369 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f31ccd-6eef-4ae8-96bb-7c0b9cb5e71b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:56.465668 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f31ccd-6eef-4ae8-96bb-7c0b9cb5e71b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:56.465950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f31ccd-6eef-4ae8-96bb-7c0b9cb5e71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:56.484523 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a5f31ccd-6eef-4ae8-96bb-7c0b9cb5e71b None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:56.485730 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a5f31ccd-6eef-4ae8-96bb-7c0b9cb5e71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:56.493723 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1429/2888] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:56.498417 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebc8ed-1295-4a10-aa99-eff38134de10 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:56.508677 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebc8ed-1295-4a10-aa99-eff38134de10 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:56.513184 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f26289-f2ff-467c-a264-2fa8d2914c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:56.513830 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f26289-f2ff-467c-a264-2fa8d2914c2b admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:56.514115 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f26289-f2ff-467c-a264-2fa8d2914c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:56.514402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f26289-f2ff-467c-a264-2fa8d2914c2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:56.515689 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f26289-f2ff-467c-a264-2fa8d2914c2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:55:56.516032 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f26289-f2ff-467c-a264-2fa8d2914c2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:56.517641 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f26289-f2ff-467c-a264-2fa8d2914c2b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:56.529277 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1460/2889] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:55:56.533675 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f31ccd-6eef-4ae8-96bb-7c0b9cb5e71b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:56.543201 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f31ccd-6eef-4ae8-96bb-7c0b9cb5e71b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:56.548077 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd72550-2f44-4be7-9c66-b1ec1f81909a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:56.548720 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd72550-2f44-4be7-9c66-b1ec1f81909a admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:56.549004 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd72550-2f44-4be7-9c66-b1ec1f81909a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:56.549278 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd72550-2f44-4be7-9c66-b1ec1f81909a admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:56.549961 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd72550-2f44-4be7-9c66-b1ec1f81909a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:56.551940 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd72550-2f44-4be7-9c66-b1ec1f81909a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:56.578797 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1430/2890] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:55:56 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:55:56.582874 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f26289-f2ff-467c-a264-2fa8d2914c2b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:56.592891 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f26289-f2ff-467c-a264-2fa8d2914c2b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:56.648098 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c37b0-8941-4485-b2eb-701b7a2cc563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:56.648762 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c37b0-8941-4485-b2eb-701b7a2cc563 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:56.649043 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c37b0-8941-4485-b2eb-701b7a2cc563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:56.649317 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c37b0-8941-4485-b2eb-701b7a2cc563 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:56.650017 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c37b0-8941-4485-b2eb-701b7a2cc563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:56.651943 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c37b0-8941-4485-b2eb-701b7a2cc563 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:56.673338 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1461/2891] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:55:56 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:55:56.678255 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd72550-2f44-4be7-9c66-b1ec1f81909a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:56.690115 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd72550-2f44-4be7-9c66-b1ec1f81909a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:56.697196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36667eb1-626c-46d8-8f35-d9708ba7d662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:56.697196 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36667eb1-626c-46d8-8f35-d9708ba7d662 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:56.700325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36667eb1-626c-46d8-8f35-d9708ba7d662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:56.700325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36667eb1-626c-46d8-8f35-d9708ba7d662 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:56.700325 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36667eb1-626c-46d8-8f35-d9708ba7d662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:56.700596 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36667eb1-626c-46d8-8f35-d9708ba7d662 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:56.714907 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1431/2892] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:55:56.718785 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c37b0-8941-4485-b2eb-701b7a2cc563 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:56.729548 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c37b0-8941-4485-b2eb-701b7a2cc563 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:56.735002 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23308551-b8ad-475e-9cc1-48dc491b3d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:56.735656 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23308551-b8ad-475e-9cc1-48dc491b3d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:56.735960 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23308551-b8ad-475e-9cc1-48dc491b3d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:56.736238 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23308551-b8ad-475e-9cc1-48dc491b3d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:56.736962 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23308551-b8ad-475e-9cc1-48dc491b3d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:56.738600 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23308551-b8ad-475e-9cc1-48dc491b3d24 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:56.750139 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1462/2893] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 09:55:56.755232 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36667eb1-626c-46d8-8f35-d9708ba7d662 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:56.766416 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36667eb1-626c-46d8-8f35-d9708ba7d662 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:56.771217 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354cab8-d5a5-4532-8baa-59367491c62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:56.772148 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354cab8-d5a5-4532-8baa-59367491c62b admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:56.772436 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354cab8-d5a5-4532-8baa-59367491c62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:56.772712 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354cab8-d5a5-4532-8baa-59367491c62b admin admin] PATH_INFO: `/v3/projects/056d05e564974474bf452336ea89f792/users/d1e4847bb19442b7b189eb3c2d1625b2/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:56.787497 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0354cab8-d5a5-4532-8baa-59367491c62b admin admin] RBAC: Authorizing `identity:create_grant(project_id=056d05e564974474bf452336ea89f792, user_id=d1e4847bb19442b7b189eb3c2d1625b2, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:56.790199 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0354cab8-d5a5-4532-8baa-59367491c62b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:56.803474 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1432/2894] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:55:56 2022] PUT /identity/v3/projects/056d05e564974474bf452336ea89f792/users/d1e4847bb19442b7b189eb3c2d1625b2/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:56.814442 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da65a32-701f-430a-bf12-e232e609c956 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:56.814950 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da65a32-701f-430a-bf12-e232e609c956 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:56.815040 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da65a32-701f-430a-bf12-e232e609c956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:56.840878 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-9da65a32-701f-430a-bf12-e232e609c956 None None] MFA Rules not processed for user `d1e4847bb19442b7b189eb3c2d1625b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:56.842071 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9da65a32-701f-430a-bf12-e232e609c956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:56.904978 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1463/2895] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:56.909504 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354cab8-d5a5-4532-8baa-59367491c62b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:56.958440 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:56.973982 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354cab8-d5a5-4532-8baa-59367491c62b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:56.978711 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b4579-7fc7-4ef5-8cf7-a211a26e7473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:56.979367 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b4579-7fc7-4ef5-8cf7-a211a26e7473 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:56.979651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b4579-7fc7-4ef5-8cf7-a211a26e7473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:56.979953 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b4579-7fc7-4ef5-8cf7-a211a26e7473 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:56.980651 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b4579-7fc7-4ef5-8cf7-a211a26e7473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:56.982585 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b4579-7fc7-4ef5-8cf7-a211a26e7473 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:57.028132 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1433/2896] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:55:56 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:55:57.040245 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da65a32-701f-430a-bf12-e232e609c956 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:57.046177 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:57.053266 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da65a32-701f-430a-bf12-e232e609c956 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:57.112978 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53652f09-c6cf-412e-a3e4-bd83423174e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:57.113647 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53652f09-c6cf-412e-a3e4-bd83423174e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:57.113936 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53652f09-c6cf-412e-a3e4-bd83423174e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:57.114212 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53652f09-c6cf-412e-a3e4-bd83423174e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:57.114913 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53652f09-c6cf-412e-a3e4-bd83423174e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:57.116888 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53652f09-c6cf-412e-a3e4-bd83423174e1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:57.137599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1464/2897] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:55:57 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:55:57.142502 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b4579-7fc7-4ef5-8cf7-a211a26e7473 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:57.152832 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b4579-7fc7-4ef5-8cf7-a211a26e7473 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:57.157944 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b914512-44f7-4669-af6c-d6df31968606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:57.158537 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b914512-44f7-4669-af6c-d6df31968606 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:57.158823 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b914512-44f7-4669-af6c-d6df31968606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:57.159099 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b914512-44f7-4669-af6c-d6df31968606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:57.159849 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b914512-44f7-4669-af6c-d6df31968606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:57.161546 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b914512-44f7-4669-af6c-d6df31968606 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:57.173599 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1434/2898] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:55:57.177607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53652f09-c6cf-412e-a3e4-bd83423174e1 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:57.188551 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53652f09-c6cf-412e-a3e4-bd83423174e1 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:57.193152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fc126-31b8-4afc-bd84-0c31f6491ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:57.193809 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fc126-31b8-4afc-bd84-0c31f6491ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:57.194087 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fc126-31b8-4afc-bd84-0c31f6491ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:57.194360 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fc126-31b8-4afc-bd84-0c31f6491ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:57.195078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3fc126-31b8-4afc-bd84-0c31f6491ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:57.196741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3fc126-31b8-4afc-bd84-0c31f6491ff9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:57.207438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1465/2899] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:55:57.211538 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b914512-44f7-4669-af6c-d6df31968606 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:57.221688 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b914512-44f7-4669-af6c-d6df31968606 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:57.226238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61713a-7f94-4eb5-a0a6-a47e5a8cf7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:57.227100 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61713a-7f94-4eb5-a0a6-a47e5a8cf7c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:57.227393 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61713a-7f94-4eb5-a0a6-a47e5a8cf7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:57.227672 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61713a-7f94-4eb5-a0a6-a47e5a8cf7c4 admin admin] PATH_INFO: `/v3/projects/8e767a890a0e4e0f923ba58bc8f84912/users/c4f14c2c994f447699b99d467b3d845c/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:57.242669 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61713a-7f94-4eb5-a0a6-a47e5a8cf7c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e767a890a0e4e0f923ba58bc8f84912, user_id=c4f14c2c994f447699b99d467b3d845c, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:57.245398 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61713a-7f94-4eb5-a0a6-a47e5a8cf7c4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:57.258858 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1435/2900] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:55:57 2022] PUT /identity/v3/projects/8e767a890a0e4e0f923ba58bc8f84912/users/c4f14c2c994f447699b99d467b3d845c/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:57.271328 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca667ae5-334b-49d7-9ab8-1cd3df86e9c8 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:57.271626 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca667ae5-334b-49d7-9ab8-1cd3df86e9c8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:57.271928 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca667ae5-334b-49d7-9ab8-1cd3df86e9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:57.424739 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ca667ae5-334b-49d7-9ab8-1cd3df86e9c8 None None] MFA Rules not processed for user `c4f14c2c994f447699b99d467b3d845c`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:57.425880 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ca667ae5-334b-49d7-9ab8-1cd3df86e9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:57.490218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1466/2901] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:57.494830 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61713a-7f94-4eb5-a0a6-a47e5a8cf7c4 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:57.558452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61713a-7f94-4eb5-a0a6-a47e5a8cf7c4 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:57.563136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbb26e-7432-4e18-80f1-2692dc49c0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:57.563787 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbb26e-7432-4e18-80f1-2692dc49c0a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:57.564116 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbb26e-7432-4e18-80f1-2692dc49c0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:57.564385 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbb26e-7432-4e18-80f1-2692dc49c0a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:57.565080 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbb26e-7432-4e18-80f1-2692dc49c0a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:57.567079 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbb26e-7432-4e18-80f1-2692dc49c0a6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:57.593568 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1436/2902] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:55:57 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:55:57.597788 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca667ae5-334b-49d7-9ab8-1cd3df86e9c8 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:57.608865 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca667ae5-334b-49d7-9ab8-1cd3df86e9c8 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:57.666326 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e44235-376b-4953-909c-d0df0d4d0233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:57.667241 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e44235-376b-4953-909c-d0df0d4d0233 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:57.667522 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e44235-376b-4953-909c-d0df0d4d0233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:57.667796 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e44235-376b-4953-909c-d0df0d4d0233 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:57.668525 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e44235-376b-4953-909c-d0df0d4d0233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:57.670425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e44235-376b-4953-909c-d0df0d4d0233 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:57.686739 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:57.694676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1467/2903] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:55:57 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:55:57.696628 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbb26e-7432-4e18-80f1-2692dc49c0a6 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:57.708179 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbb26e-7432-4e18-80f1-2692dc49c0a6 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:57.718410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2b431-de52-46a6-9990-7cce4e724612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:57.719139 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2b431-de52-46a6-9990-7cce4e724612 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:57.719429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2b431-de52-46a6-9990-7cce4e724612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:57.719713 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c2b431-de52-46a6-9990-7cce4e724612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:57.720479 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2b431-de52-46a6-9990-7cce4e724612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:57.722156 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c2b431-de52-46a6-9990-7cce4e724612 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:57.733915 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1437/2904] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:55:57.738078 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e44235-376b-4953-909c-d0df0d4d0233 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:57.748202 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e44235-376b-4953-909c-d0df0d4d0233 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:57.753411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f06b27-583a-4a25-98d7-ab16ed978b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:57.754288 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f06b27-583a-4a25-98d7-ab16ed978b94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:57.754569 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f06b27-583a-4a25-98d7-ab16ed978b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:57.754841 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f06b27-583a-4a25-98d7-ab16ed978b94 admin admin] PATH_INFO: `/v3/projects/ba317a652cae457f94e8fb022a47a2a7/users/b3eae6b2fc084aeab4cfe3ca5f9f338c/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:57.770354 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f06b27-583a-4a25-98d7-ab16ed978b94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba317a652cae457f94e8fb022a47a2a7, user_id=b3eae6b2fc084aeab4cfe3ca5f9f338c, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:57.773045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f06b27-583a-4a25-98d7-ab16ed978b94 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:57.785589 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1468/2905] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:55:57 2022] PUT /identity/v3/projects/ba317a652cae457f94e8fb022a47a2a7/users/b3eae6b2fc084aeab4cfe3ca5f9f338c/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:57.789627 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2b431-de52-46a6-9990-7cce4e724612 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:57.800140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c2b431-de52-46a6-9990-7cce4e724612 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:57.855727 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2e06c-8869-4b2b-a79f-1ff1f4161606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:57.856408 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2e06c-8869-4b2b-a79f-1ff1f4161606 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:57.856691 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2e06c-8869-4b2b-a79f-1ff1f4161606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:57.856961 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2e06c-8869-4b2b-a79f-1ff1f4161606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:57.857671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2e06c-8869-4b2b-a79f-1ff1f4161606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:57.860594 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2e06c-8869-4b2b-a79f-1ff1f4161606 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:57.872471 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1438/2906] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:55:57.877217 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f06b27-583a-4a25-98d7-ab16ed978b94 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:57.887379 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f06b27-583a-4a25-98d7-ab16ed978b94 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:57.896117 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdd7b1-99f6-4599-9aa2-bed86c88e7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:57.897072 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdd7b1-99f6-4599-9aa2-bed86c88e7f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:57.897353 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdd7b1-99f6-4599-9aa2-bed86c88e7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:57.897626 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdd7b1-99f6-4599-9aa2-bed86c88e7f1 admin admin] PATH_INFO: `/v3/projects/ba317a652cae457f94e8fb022a47a2a7/users/b3eae6b2fc084aeab4cfe3ca5f9f338c/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:57.899653 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bdd7b1-99f6-4599-9aa2-bed86c88e7f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba317a652cae457f94e8fb022a47a2a7, user_id=b3eae6b2fc084aeab4cfe3ca5f9f338c, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:57.902331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bdd7b1-99f6-4599-9aa2-bed86c88e7f1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:57.912885 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1469/2907] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:55:57 2022] PUT /identity/v3/projects/ba317a652cae457f94e8fb022a47a2a7/users/b3eae6b2fc084aeab4cfe3ca5f9f338c/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:55:57.923604 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b4799-af33-4786-a21f-657c056bc19a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:57.923901 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b4799-af33-4786-a21f-657c056bc19a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:57.924215 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b4799-af33-4786-a21f-657c056bc19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:57.943013 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:55:57.953976 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f84b4799-af33-4786-a21f-657c056bc19a None None] MFA Rules not processed for user `b3eae6b2fc084aeab4cfe3ca5f9f338c`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:55:57.955187 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f84b4799-af33-4786-a21f-657c056bc19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:58.023307 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1439/2908] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:55:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:55:58.034080 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdd7b1-99f6-4599-9aa2-bed86c88e7f1 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:58.035761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-90bdd7b1-99f6-4599-9aa2-bed86c88e7f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:58.094140 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdd7b1-99f6-4599-9aa2-bed86c88e7f1 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:58.097795 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316115d-b7b0-498b-9f23-c133feb0c1ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:58.098424 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316115d-b7b0-498b-9f23-c133feb0c1ea service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:58.098703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316115d-b7b0-498b-9f23-c133feb0c1ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:58.098976 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316115d-b7b0-498b-9f23-c133feb0c1ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:58.108565 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316115d-b7b0-498b-9f23-c133feb0c1ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:58.109997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316115d-b7b0-498b-9f23-c133feb0c1ea service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:58.119182 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1470/2909] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:55:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:55:58.618401 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b4799-af33-4786-a21f-657c056bc19a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:55:58.619819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f84b4799-af33-4786-a21f-657c056bc19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:55:58.632826 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b4799-af33-4786-a21f-657c056bc19a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:55:58.636444 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92fcde-b3db-4c53-85ac-33720b9d29ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:55:58.637091 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92fcde-b3db-4c53-85ac-33720b9d29ea service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:55:58.637375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92fcde-b3db-4c53-85ac-33720b9d29ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:55:58.637652 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92fcde-b3db-4c53-85ac-33720b9d29ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:55:58.695621 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92fcde-b3db-4c53-85ac-33720b9d29ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:55:58.697126 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92fcde-b3db-4c53-85ac-33720b9d29ea service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:55:58.706574 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1440/2910] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:55:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:56:02.697005 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316115d-b7b0-498b-9f23-c133feb0c1ea service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:02.706271 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316115d-b7b0-498b-9f23-c133feb0c1ea service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:02.709942 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7430b7-1106-4977-8bc6-7c681567a8f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:02.710595 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7430b7-1106-4977-8bc6-7c681567a8f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:02.710877 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7430b7-1106-4977-8bc6-7c681567a8f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:02.711152 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7430b7-1106-4977-8bc6-7c681567a8f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:02.774988 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7430b7-1106-4977-8bc6-7c681567a8f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:02.776380 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7430b7-1106-4977-8bc6-7c681567a8f7 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:02.785336 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1471/2911] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:56:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:56:04.865691 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92fcde-b3db-4c53-85ac-33720b9d29ea service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:04.931162 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92fcde-b3db-4c53-85ac-33720b9d29ea service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:04.935689 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfe9ae-ba27-4fda-a054-b1950411ea05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:04.936769 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfe9ae-ba27-4fda-a054-b1950411ea05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:04.937056 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfe9ae-ba27-4fda-a054-b1950411ea05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:04.937332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfe9ae-ba27-4fda-a054-b1950411ea05 admin admin] PATH_INFO: `/v3/users/d1e4847bb19442b7b189eb3c2d1625b2` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:04.938541 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfe9ae-ba27-4fda-a054-b1950411ea05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1e4847bb19442b7b189eb3c2d1625b2)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:04.940546 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfe9ae-ba27-4fda-a054-b1950411ea05 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:04.980306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:56:04.990580 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ebbfe9ae-ba27-4fda-a054-b1950411ea05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1e4847bb19442b7b189eb3c2d1625b2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:04.998337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ebbfe9ae-ba27-4fda-a054-b1950411ea05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1e4847bb19442b7b189eb3c2d1625b2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:05.014924 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ebbfe9ae-ba27-4fda-a054-b1950411ea05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1e4847bb19442b7b189eb3c2d1625b2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:05.029052 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ebbfe9ae-ba27-4fda-a054-b1950411ea05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1e4847bb19442b7b189eb3c2d1625b2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:05.030817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ebbfe9ae-ba27-4fda-a054-b1950411ea05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1e4847bb19442b7b189eb3c2d1625b2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:05.039001 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1441/2912] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:56:04 2022] DELETE /identity/v3/users/d1e4847bb19442b7b189eb3c2d1625b2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:05.043975 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e849e-12d4-4680-95f6-8dc490535223 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:05.044432 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e849e-12d4-4680-95f6-8dc490535223 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:05.044545 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e849e-12d4-4680-95f6-8dc490535223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:05.061208 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6d5e849e-12d4-4680-95f6-8dc490535223 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:56:05.062412 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6d5e849e-12d4-4680-95f6-8dc490535223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:05.121475 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1472/2913] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:56:05.131452 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfe9ae-ba27-4fda-a054-b1950411ea05 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:05.132981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ebbfe9ae-ba27-4fda-a054-b1950411ea05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:05.191408 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfe9ae-ba27-4fda-a054-b1950411ea05 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:05.195248 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2acbb7-3c84-4669-91b2-990a28655198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:05.195885 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2acbb7-3c84-4669-91b2-990a28655198 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:05.196190 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2acbb7-3c84-4669-91b2-990a28655198 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:05.196462 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2acbb7-3c84-4669-91b2-990a28655198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:05.206150 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2acbb7-3c84-4669-91b2-990a28655198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:05.207577 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2acbb7-3c84-4669-91b2-990a28655198 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:05.216872 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1442/2914] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:56:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:56:05.483939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e849e-12d4-4680-95f6-8dc490535223 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:05.485497 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6d5e849e-12d4-4680-95f6-8dc490535223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:05.494117 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:56:05.497879 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e849e-12d4-4680-95f6-8dc490535223 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:05.504138 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:05.505043 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:05.505330 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:05.505599 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5 admin admin] PATH_INFO: `/v3/projects/056d05e564974474bf452336ea89f792` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:05.506786 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=056d05e564974474bf452336ea89f792)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:05.508768 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:05.555228 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '056d05e564974474bf452336ea89f792', 'request_id': 'req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:05.560752 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '056d05e564974474bf452336ea89f792', 'request_id': 'req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:05.567531 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '056d05e564974474bf452336ea89f792', 'request_id': 'req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:05.580333 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5 admin admin] The token cache is being invalidate because project 056d05e564974474bf452336ea89f792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:56:05.580624 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:05.581349 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:05.583399 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1473/2915] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:56:05 2022] DELETE /identity/v3/projects/056d05e564974474bf452336ea89f792 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:05.587329 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2acbb7-3c84-4669-91b2-990a28655198 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:05.597047 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2acbb7-3c84-4669-91b2-990a28655198 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:05.654498 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d538e2-b8bf-41f5-9966-76360d0b8bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:05.656142 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d538e2-b8bf-41f5-9966-76360d0b8bbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:05.656429 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d538e2-b8bf-41f5-9966-76360d0b8bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:05.656710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d538e2-b8bf-41f5-9966-76360d0b8bbc admin admin] PATH_INFO: `/v3/users/c4f14c2c994f447699b99d467b3d845c` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:05.657905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d538e2-b8bf-41f5-9966-76360d0b8bbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4f14c2c994f447699b99d467b3d845c)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:05.659994 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d538e2-b8bf-41f5-9966-76360d0b8bbc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:05.714085 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c7d538e2-b8bf-41f5-9966-76360d0b8bbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4f14c2c994f447699b99d467b3d845c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:05.719871 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c7d538e2-b8bf-41f5-9966-76360d0b8bbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4f14c2c994f447699b99d467b3d845c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:05.722223 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:56:05.730021 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c7d538e2-b8bf-41f5-9966-76360d0b8bbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4f14c2c994f447699b99d467b3d845c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:05.742996 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c7d538e2-b8bf-41f5-9966-76360d0b8bbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4f14c2c994f447699b99d467b3d845c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:05.743654 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c7d538e2-b8bf-41f5-9966-76360d0b8bbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4f14c2c994f447699b99d467b3d845c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:05.750117 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1443/2916] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:56:05 2022] DELETE /identity/v3/users/c4f14c2c994f447699b99d467b3d845c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:06.029006 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:06.030756 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:06.047773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f44fb-ddf1-486b-8c91-5396b1fa0fc5 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:06.109272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f548079-a7cf-4043-a3ea-25196cd56aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:06.110132 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f548079-a7cf-4043-a3ea-25196cd56aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:06.110418 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f548079-a7cf-4043-a3ea-25196cd56aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:06.110693 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f548079-a7cf-4043-a3ea-25196cd56aca admin admin] PATH_INFO: `/v3/projects/8e767a890a0e4e0f923ba58bc8f84912` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:06.111889 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f548079-a7cf-4043-a3ea-25196cd56aca admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e767a890a0e4e0f923ba58bc8f84912)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:06.113863 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f548079-a7cf-4043-a3ea-25196cd56aca admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:06.160853 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8f548079-a7cf-4043-a3ea-25196cd56aca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e767a890a0e4e0f923ba58bc8f84912', 'request_id': 'req-8f548079-a7cf-4043-a3ea-25196cd56aca'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:06.166419 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8f548079-a7cf-4043-a3ea-25196cd56aca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e767a890a0e4e0f923ba58bc8f84912', 'request_id': 'req-8f548079-a7cf-4043-a3ea-25196cd56aca'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:06.172761 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8f548079-a7cf-4043-a3ea-25196cd56aca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e767a890a0e4e0f923ba58bc8f84912', 'request_id': 'req-8f548079-a7cf-4043-a3ea-25196cd56aca'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:06.187732 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8f548079-a7cf-4043-a3ea-25196cd56aca admin admin] The token cache is being invalidate because project 8e767a890a0e4e0f923ba58bc8f84912 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:56:06.188054 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8f548079-a7cf-4043-a3ea-25196cd56aca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:06.188710 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8f548079-a7cf-4043-a3ea-25196cd56aca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:06.190271 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1474/2917] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:56:06 2022] DELETE /identity/v3/projects/8e767a890a0e4e0f923ba58bc8f84912 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:06.194285 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d538e2-b8bf-41f5-9966-76360d0b8bbc admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:06.195800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c7d538e2-b8bf-41f5-9966-76360d0b8bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:06.210209 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d538e2-b8bf-41f5-9966-76360d0b8bbc admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:06.266745 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfc7de-c440-43cb-a886-817288b30106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:06.267614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc7de-c440-43cb-a886-817288b30106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:06.267902 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc7de-c440-43cb-a886-817288b30106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:06.268214 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfc7de-c440-43cb-a886-817288b30106 admin admin] PATH_INFO: `/v3/users/b3eae6b2fc084aeab4cfe3ca5f9f338c` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:06.269414 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfc7de-c440-43cb-a886-817288b30106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3eae6b2fc084aeab4cfe3ca5f9f338c)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:06.271381 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfc7de-c440-43cb-a886-817288b30106 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:06.318349 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-60dfc7de-c440-43cb-a886-817288b30106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3eae6b2fc084aeab4cfe3ca5f9f338c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:06.325739 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-60dfc7de-c440-43cb-a886-817288b30106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3eae6b2fc084aeab4cfe3ca5f9f338c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:06.333920 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-60dfc7de-c440-43cb-a886-817288b30106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3eae6b2fc084aeab4cfe3ca5f9f338c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:06.345904 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-60dfc7de-c440-43cb-a886-817288b30106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3eae6b2fc084aeab4cfe3ca5f9f338c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:06.346557 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-60dfc7de-c440-43cb-a886-817288b30106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3eae6b2fc084aeab4cfe3ca5f9f338c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:06.352536 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1444/2918] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:56:06 2022] DELETE /identity/v3/users/b3eae6b2fc084aeab4cfe3ca5f9f338c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:06.634977 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f548079-a7cf-4043-a3ea-25196cd56aca admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:06.636564 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8f548079-a7cf-4043-a3ea-25196cd56aca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:06.649739 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f548079-a7cf-4043-a3ea-25196cd56aca admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:06.665632 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:56:06.723371 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014e37b-f091-41e4-928e-eafbc44f2389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:06.724392 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014e37b-f091-41e4-928e-eafbc44f2389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:06.724672 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014e37b-f091-41e4-928e-eafbc44f2389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:06.724945 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014e37b-f091-41e4-928e-eafbc44f2389 admin admin] PATH_INFO: `/v3/projects/ba317a652cae457f94e8fb022a47a2a7` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:06.726265 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f014e37b-f091-41e4-928e-eafbc44f2389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba317a652cae457f94e8fb022a47a2a7)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:06.728221 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f014e37b-f091-41e4-928e-eafbc44f2389 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:06.778411 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f014e37b-f091-41e4-928e-eafbc44f2389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba317a652cae457f94e8fb022a47a2a7', 'request_id': 'req-f014e37b-f091-41e4-928e-eafbc44f2389'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:06.784566 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f014e37b-f091-41e4-928e-eafbc44f2389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba317a652cae457f94e8fb022a47a2a7', 'request_id': 'req-f014e37b-f091-41e4-928e-eafbc44f2389'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:06.791703 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f014e37b-f091-41e4-928e-eafbc44f2389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba317a652cae457f94e8fb022a47a2a7', 'request_id': 'req-f014e37b-f091-41e4-928e-eafbc44f2389'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:06.806458 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f014e37b-f091-41e4-928e-eafbc44f2389 admin admin] The token cache is being invalidate because project ba317a652cae457f94e8fb022a47a2a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:56:06.806774 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f014e37b-f091-41e4-928e-eafbc44f2389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:06.807440 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f014e37b-f091-41e4-928e-eafbc44f2389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:06.809037 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1475/2919] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:56:06 2022] DELETE /identity/v3/projects/ba317a652cae457f94e8fb022a47a2a7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:06.814962 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736310bb-fe3c-4bf5-a281-da79e041095a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:06.815276 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736310bb-fe3c-4bf5-a281-da79e041095a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:06.815546 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736310bb-fe3c-4bf5-a281-da79e041095a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:06.833517 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-736310bb-fe3c-4bf5-a281-da79e041095a None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:56:06.834644 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-736310bb-fe3c-4bf5-a281-da79e041095a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:06.881042 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:56:06.894979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1445/2920] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:56:06.903317 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae9862-4302-4d6b-9680-ef304b0dbfad None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:06.903633 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae9862-4302-4d6b-9680-ef304b0dbfad None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:06.903939 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae9862-4302-4d6b-9680-ef304b0dbfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:06.923773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-15ae9862-4302-4d6b-9680-ef304b0dbfad None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:56:06.924998 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-15ae9862-4302-4d6b-9680-ef304b0dbfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:06.933471 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1476/2921] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:56:06.937832 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736310bb-fe3c-4bf5-a281-da79e041095a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:06.948334 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736310bb-fe3c-4bf5-a281-da79e041095a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:06.953028 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc51ab6-8bf9-42ef-86e3-e371a16c81fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:06.953680 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc51ab6-8bf9-42ef-86e3-e371a16c81fb admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:06.953963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc51ab6-8bf9-42ef-86e3-e371a16c81fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:06.954239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc51ab6-8bf9-42ef-86e3-e371a16c81fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:06.954963 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc51ab6-8bf9-42ef-86e3-e371a16c81fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:56:06.955270 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc51ab6-8bf9-42ef-86e3-e371a16c81fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:06.956923 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc51ab6-8bf9-42ef-86e3-e371a16c81fb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:06.969188 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1446/2922] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:56:06.973562 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae9862-4302-4d6b-9680-ef304b0dbfad None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:06.982760 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae9862-4302-4d6b-9680-ef304b0dbfad None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:06.988570 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3edac-b4f6-427c-b0e6-055c50618c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:06.989211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3edac-b4f6-427c-b0e6-055c50618c80 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:06.989495 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3edac-b4f6-427c-b0e6-055c50618c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:06.989767 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3edac-b4f6-427c-b0e6-055c50618c80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:06.990454 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3edac-b4f6-427c-b0e6-055c50618c80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:06.992402 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3edac-b4f6-427c-b0e6-055c50618c80 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:07.018255 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1477/2923] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:56:06 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:56:07.022435 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc51ab6-8bf9-42ef-86e3-e371a16c81fb admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:07.033139 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc51ab6-8bf9-42ef-86e3-e371a16c81fb admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:07.090025 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df466e77-3b6b-4b5e-885a-0cf1e5ef728d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:07.090589 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df466e77-3b6b-4b5e-885a-0cf1e5ef728d admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:07.090883 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df466e77-3b6b-4b5e-885a-0cf1e5ef728d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:07.091173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df466e77-3b6b-4b5e-885a-0cf1e5ef728d admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:07.091863 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df466e77-3b6b-4b5e-885a-0cf1e5ef728d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:07.094410 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df466e77-3b6b-4b5e-885a-0cf1e5ef728d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:07.115296 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1447/2924] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:56:07 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:56:07.119455 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3edac-b4f6-427c-b0e6-055c50618c80 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:07.129631 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3edac-b4f6-427c-b0e6-055c50618c80 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:07.134216 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac141c39-432a-4918-9747-c732972929ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:07.134864 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac141c39-432a-4918-9747-c732972929ab admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:07.135144 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac141c39-432a-4918-9747-c732972929ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:07.135542 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac141c39-432a-4918-9747-c732972929ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:07.136294 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac141c39-432a-4918-9747-c732972929ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:07.137876 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac141c39-432a-4918-9747-c732972929ab admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:07.148900 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1478/2925] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:56:07.153015 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df466e77-3b6b-4b5e-885a-0cf1e5ef728d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:07.163800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df466e77-3b6b-4b5e-885a-0cf1e5ef728d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:07.168791 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f858fe06-3ac2-4069-b939-8f4b03ed4950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:07.169449 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858fe06-3ac2-4069-b939-8f4b03ed4950 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:07.169739 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858fe06-3ac2-4069-b939-8f4b03ed4950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:07.170017 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858fe06-3ac2-4069-b939-8f4b03ed4950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:07.170778 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f858fe06-3ac2-4069-b939-8f4b03ed4950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:07.172484 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f858fe06-3ac2-4069-b939-8f4b03ed4950 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:07.182470 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1448/2926] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:56:07.187655 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac141c39-432a-4918-9747-c732972929ab admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:07.198176 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac141c39-432a-4918-9747-c732972929ab admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:07.204044 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa831c-b747-461e-bf32-cfb709d4fda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:07.204925 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa831c-b747-461e-bf32-cfb709d4fda1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:07.205214 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa831c-b747-461e-bf32-cfb709d4fda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:07.205492 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa831c-b747-461e-bf32-cfb709d4fda1 admin admin] PATH_INFO: `/v3/projects/905acb247519436a8d39a16551a30588/users/76296da07f4a478ea48f18b5bf2906df/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:07.221297 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa831c-b747-461e-bf32-cfb709d4fda1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=905acb247519436a8d39a16551a30588, user_id=76296da07f4a478ea48f18b5bf2906df, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:07.223973 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa831c-b747-461e-bf32-cfb709d4fda1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:07.238069 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1479/2927] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:56:07 2022] PUT /identity/v3/projects/905acb247519436a8d39a16551a30588/users/76296da07f4a478ea48f18b5bf2906df/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:07.250788 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7f733-d2b4-4db3-b8f6-25727918897c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:07.251435 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7f733-d2b4-4db3-b8f6-25727918897c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:07.251435 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7f733-d2b4-4db3-b8f6-25727918897c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:07.279375 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e3a7f733-d2b4-4db3-b8f6-25727918897c None None] MFA Rules not processed for user `76296da07f4a478ea48f18b5bf2906df`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:56:07.281311 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e3a7f733-d2b4-4db3-b8f6-25727918897c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:07.346305 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1449/2928] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:56:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 09:56:07.350759 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa831c-b747-461e-bf32-cfb709d4fda1 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:07.396691 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:56:07.415103 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa831c-b747-461e-bf32-cfb709d4fda1 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:07.420067 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080deed-3b5d-46e3-8b89-a2d1fcc920a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:07.420744 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080deed-3b5d-46e3-8b89-a2d1fcc920a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:07.421025 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080deed-3b5d-46e3-8b89-a2d1fcc920a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:07.421299 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080deed-3b5d-46e3-8b89-a2d1fcc920a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:07.421995 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080deed-3b5d-46e3-8b89-a2d1fcc920a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:07.423884 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080deed-3b5d-46e3-8b89-a2d1fcc920a8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:07.448882 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1480/2929] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:56:07 2022] POST /identity/v3/projects => generated 393 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:56:07.453850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7f733-d2b4-4db3-b8f6-25727918897c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:07.464905 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7f733-d2b4-4db3-b8f6-25727918897c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:07.511472 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:56:07.525533 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca2cea-678a-4975-962a-69e2efcf3bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:07.526205 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca2cea-678a-4975-962a-69e2efcf3bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:07.526490 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca2cea-678a-4975-962a-69e2efcf3bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:07.526768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca2cea-678a-4975-962a-69e2efcf3bc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:07.527475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca2cea-678a-4975-962a-69e2efcf3bc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:07.529447 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca2cea-678a-4975-962a-69e2efcf3bc5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:07.549366 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1450/2930] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:56:07 2022] POST /identity/v3/users => generated 303 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:56:07.553415 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080deed-3b5d-46e3-8b89-a2d1fcc920a8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:07.564865 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080deed-3b5d-46e3-8b89-a2d1fcc920a8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:07.570047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82dbde-4809-4b44-a75e-46a24bfc8de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:07.570697 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82dbde-4809-4b44-a75e-46a24bfc8de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:07.570999 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82dbde-4809-4b44-a75e-46a24bfc8de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:07.571270 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82dbde-4809-4b44-a75e-46a24bfc8de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:07.572024 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82dbde-4809-4b44-a75e-46a24bfc8de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:07.573609 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82dbde-4809-4b44-a75e-46a24bfc8de5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:07.585879 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1481/2931] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:56:07.590460 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca2cea-678a-4975-962a-69e2efcf3bc5 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:07.600784 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca2cea-678a-4975-962a-69e2efcf3bc5 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:07.605667 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2b657-96eb-4d4f-b6c9-70bd80cd8c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:07.606536 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2b657-96eb-4d4f-b6c9-70bd80cd8c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:07.606815 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2b657-96eb-4d4f-b6c9-70bd80cd8c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:07.607086 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2b657-96eb-4d4f-b6c9-70bd80cd8c7d admin admin] PATH_INFO: `/v3/projects/3653c18f26474021a165ad3573ea59fa/users/9adb37bc5d9d40ba8d019b56e4ff1b0e/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:07.622200 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2b657-96eb-4d4f-b6c9-70bd80cd8c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3653c18f26474021a165ad3573ea59fa, user_id=9adb37bc5d9d40ba8d019b56e4ff1b0e, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:07.625418 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2b657-96eb-4d4f-b6c9-70bd80cd8c7d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:07.640173 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1451/2932] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:56:07 2022] PUT /identity/v3/projects/3653c18f26474021a165ad3573ea59fa/users/9adb37bc5d9d40ba8d019b56e4ff1b0e/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:07.644165 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82dbde-4809-4b44-a75e-46a24bfc8de5 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:07.654178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82dbde-4809-4b44-a75e-46a24bfc8de5 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:07.714481 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33558e4c-57a3-48d1-a46e-2d9d9769395e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:07.715109 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33558e4c-57a3-48d1-a46e-2d9d9769395e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:07.715397 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33558e4c-57a3-48d1-a46e-2d9d9769395e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:07.715678 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33558e4c-57a3-48d1-a46e-2d9d9769395e admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:07.716461 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33558e4c-57a3-48d1-a46e-2d9d9769395e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:07.718196 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33558e4c-57a3-48d1-a46e-2d9d9769395e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:07.728992 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1482/2933] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:56:07.733113 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2b657-96eb-4d4f-b6c9-70bd80cd8c7d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:07.744728 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2b657-96eb-4d4f-b6c9-70bd80cd8c7d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:07.750588 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec76cc-2454-4c8e-922b-d79f00f68239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:07.751481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec76cc-2454-4c8e-922b-d79f00f68239 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:07.751768 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec76cc-2454-4c8e-922b-d79f00f68239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:07.752076 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec76cc-2454-4c8e-922b-d79f00f68239 admin admin] PATH_INFO: `/v3/projects/3653c18f26474021a165ad3573ea59fa/users/9adb37bc5d9d40ba8d019b56e4ff1b0e/roles/000e942517f2469ba36d2e65978bc314` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:07.754090 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec76cc-2454-4c8e-922b-d79f00f68239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3653c18f26474021a165ad3573ea59fa, user_id=9adb37bc5d9d40ba8d019b56e4ff1b0e, role_id=000e942517f2469ba36d2e65978bc314)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:07.756811 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec76cc-2454-4c8e-922b-d79f00f68239 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:07.769069 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1452/2934] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:56:07 2022] PUT /identity/v3/projects/3653c18f26474021a165ad3573ea59fa/users/9adb37bc5d9d40ba8d019b56e4ff1b0e/roles/000e942517f2469ba36d2e65978bc314 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:07.779667 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2b3de-dd95-4a2c-8209-af48af308bec None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:07.779996 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2b3de-dd95-4a2c-8209-af48af308bec None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:07.780306 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2b3de-dd95-4a2c-8209-af48af308bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:07.806753 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c3e2b3de-dd95-4a2c-8209-af48af308bec None None] MFA Rules not processed for user `9adb37bc5d9d40ba8d019b56e4ff1b0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:56:07.808017 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c3e2b3de-dd95-4a2c-8209-af48af308bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:07.877209 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1483/2935] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:56:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:56:07.887979 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec76cc-2454-4c8e-922b-d79f00f68239 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:07.889749 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f0ec76cc-2454-4c8e-922b-d79f00f68239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:07.947182 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec76cc-2454-4c8e-922b-d79f00f68239 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:07.950823 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922fa0e-838a-438f-8e5b-065095804a91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:07.951506 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922fa0e-838a-438f-8e5b-065095804a91 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:07.951816 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922fa0e-838a-438f-8e5b-065095804a91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:07.952140 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922fa0e-838a-438f-8e5b-065095804a91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:07.961311 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922fa0e-838a-438f-8e5b-065095804a91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:07.962817 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922fa0e-838a-438f-8e5b-065095804a91 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:07.972401 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1453/2936] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:56:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:56:08.019076 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2b3de-dd95-4a2c-8209-af48af308bec None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:08.020481 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c3e2b3de-dd95-4a2c-8209-af48af308bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:08.032441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2b3de-dd95-4a2c-8209-af48af308bec None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:08.036097 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8a9ad-5439-4792-b894-c80ce58d9bec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:08.036757 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8a9ad-5439-4792-b894-c80ce58d9bec service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:08.037047 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8a9ad-5439-4792-b894-c80ce58d9bec service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:08.037323 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8a9ad-5439-4792-b894-c80ce58d9bec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:08.079510 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:56:08.098607 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8a9ad-5439-4792-b894-c80ce58d9bec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:08.100083 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8a9ad-5439-4792-b894-c80ce58d9bec service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:08.109478 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1484/2937] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:56:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:56:11.384118 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922fa0e-838a-438f-8e5b-065095804a91 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:11.448554 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922fa0e-838a-438f-8e5b-065095804a91 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:11.453710 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f282da-f2fe-490f-a583-0ed44ee7e790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:11.454572 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f282da-f2fe-490f-a583-0ed44ee7e790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:11.454855 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f282da-f2fe-490f-a583-0ed44ee7e790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:11.455131 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f282da-f2fe-490f-a583-0ed44ee7e790 admin admin] PATH_INFO: `/v3/users/76296da07f4a478ea48f18b5bf2906df` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:11.456337 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f282da-f2fe-490f-a583-0ed44ee7e790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76296da07f4a478ea48f18b5bf2906df)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:11.458581 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f282da-f2fe-490f-a583-0ed44ee7e790 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:11.510304 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:56:11.511863 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a0f282da-f2fe-490f-a583-0ed44ee7e790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76296da07f4a478ea48f18b5bf2906df'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:11.521123 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a0f282da-f2fe-490f-a583-0ed44ee7e790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76296da07f4a478ea48f18b5bf2906df'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:11.530514 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a0f282da-f2fe-490f-a583-0ed44ee7e790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76296da07f4a478ea48f18b5bf2906df'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:11.543218 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a0f282da-f2fe-490f-a583-0ed44ee7e790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76296da07f4a478ea48f18b5bf2906df'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:11.543895 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a0f282da-f2fe-490f-a583-0ed44ee7e790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76296da07f4a478ea48f18b5bf2906df'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:11.550631 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1454/2938] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:56:11 2022] DELETE /identity/v3/users/76296da07f4a478ea48f18b5bf2906df => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:11.555336 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afcc69-412d-4a39-92ab-776fdcc26a6f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:11.555793 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afcc69-412d-4a39-92ab-776fdcc26a6f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:11.556118 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afcc69-412d-4a39-92ab-776fdcc26a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:11.573068 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d9afcc69-412d-4a39-92ab-776fdcc26a6f None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:56:11.574272 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d9afcc69-412d-4a39-92ab-776fdcc26a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:11.633141 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1485/2939] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:56:11.643614 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f282da-f2fe-490f-a583-0ed44ee7e790 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:11.645054 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a0f282da-f2fe-490f-a583-0ed44ee7e790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:11.702848 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f282da-f2fe-490f-a583-0ed44ee7e790 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:11.706483 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc9360-8a74-413c-8b85-fc8030a49c1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:11.707121 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc9360-8a74-413c-8b85-fc8030a49c1f service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:11.707400 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc9360-8a74-413c-8b85-fc8030a49c1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:11.707671 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc9360-8a74-413c-8b85-fc8030a49c1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:11.718856 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc9360-8a74-413c-8b85-fc8030a49c1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:11.720377 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc9360-8a74-413c-8b85-fc8030a49c1f service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:11.730850 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1455/2940] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:56:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:56:12.007894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afcc69-412d-4a39-92ab-776fdcc26a6f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:12.009322 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d9afcc69-412d-4a39-92ab-776fdcc26a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:12.025451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afcc69-412d-4a39-92ab-776fdcc26a6f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:12.030372 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8894f-c7a1-4cac-98d0-52223017e7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:12.031247 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8894f-c7a1-4cac-98d0-52223017e7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:12.031540 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8894f-c7a1-4cac-98d0-52223017e7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:12.031816 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8894f-c7a1-4cac-98d0-52223017e7c2 admin admin] PATH_INFO: `/v3/projects/905acb247519436a8d39a16551a30588` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:12.033147 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8894f-c7a1-4cac-98d0-52223017e7c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=905acb247519436a8d39a16551a30588)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:12.035051 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8894f-c7a1-4cac-98d0-52223017e7c2 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:12.085797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-afb8894f-c7a1-4cac-98d0-52223017e7c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '905acb247519436a8d39a16551a30588', 'request_id': 'req-afb8894f-c7a1-4cac-98d0-52223017e7c2'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:12.091901 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-afb8894f-c7a1-4cac-98d0-52223017e7c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '905acb247519436a8d39a16551a30588', 'request_id': 'req-afb8894f-c7a1-4cac-98d0-52223017e7c2'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:12.107139 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-afb8894f-c7a1-4cac-98d0-52223017e7c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '905acb247519436a8d39a16551a30588', 'request_id': 'req-afb8894f-c7a1-4cac-98d0-52223017e7c2'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:12.121307 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-afb8894f-c7a1-4cac-98d0-52223017e7c2 admin admin] The token cache is being invalidate because project 905acb247519436a8d39a16551a30588 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:56:12.121593 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-afb8894f-c7a1-4cac-98d0-52223017e7c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:12.122211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-afb8894f-c7a1-4cac-98d0-52223017e7c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:12.124815 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1486/2941] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:56:12 2022] DELETE /identity/v3/projects/905acb247519436a8d39a16551a30588 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:12.128796 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc9360-8a74-413c-8b85-fc8030a49c1f service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:12.138332 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc9360-8a74-413c-8b85-fc8030a49c1f service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:12.195267 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674aa36f-0c20-42b2-b52e-2ea919050801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:12.196189 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674aa36f-0c20-42b2-b52e-2ea919050801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:12.196477 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674aa36f-0c20-42b2-b52e-2ea919050801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:12.196757 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674aa36f-0c20-42b2-b52e-2ea919050801 admin admin] PATH_INFO: `/v3/users/9adb37bc5d9d40ba8d019b56e4ff1b0e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:12.197959 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674aa36f-0c20-42b2-b52e-2ea919050801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9adb37bc5d9d40ba8d019b56e4ff1b0e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:12.199937 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674aa36f-0c20-42b2-b52e-2ea919050801 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:12.245526 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-674aa36f-0c20-42b2-b52e-2ea919050801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9adb37bc5d9d40ba8d019b56e4ff1b0e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:12.253828 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-674aa36f-0c20-42b2-b52e-2ea919050801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9adb37bc5d9d40ba8d019b56e4ff1b0e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:12.261267 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-674aa36f-0c20-42b2-b52e-2ea919050801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9adb37bc5d9d40ba8d019b56e4ff1b0e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:12.266516 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:56:12.278876 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-674aa36f-0c20-42b2-b52e-2ea919050801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9adb37bc5d9d40ba8d019b56e4ff1b0e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:12.280238 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-674aa36f-0c20-42b2-b52e-2ea919050801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9adb37bc5d9d40ba8d019b56e4ff1b0e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:12.286674 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1456/2942] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:56:12 2022] DELETE /identity/v3/users/9adb37bc5d9d40ba8d019b56e4ff1b0e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:12.565188 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8894f-c7a1-4cac-98d0-52223017e7c2 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:12.566826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-afb8894f-c7a1-4cac-98d0-52223017e7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:12.582965 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8894f-c7a1-4cac-98d0-52223017e7c2 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:12.642859 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56a53f-86d3-43c5-89a7-542506a74942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:12.643718 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56a53f-86d3-43c5-89a7-542506a74942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:12.644155 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56a53f-86d3-43c5-89a7-542506a74942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:12.644430 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56a53f-86d3-43c5-89a7-542506a74942 admin admin] PATH_INFO: `/v3/projects/3653c18f26474021a165ad3573ea59fa` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:12.645610 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56a53f-86d3-43c5-89a7-542506a74942 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3653c18f26474021a165ad3573ea59fa)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:12.647509 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56a53f-86d3-43c5-89a7-542506a74942 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:12.649833 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:56:12.696441 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-df56a53f-86d3-43c5-89a7-542506a74942 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3653c18f26474021a165ad3573ea59fa', 'request_id': 'req-df56a53f-86d3-43c5-89a7-542506a74942'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:12.702337 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-df56a53f-86d3-43c5-89a7-542506a74942 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3653c18f26474021a165ad3573ea59fa', 'request_id': 'req-df56a53f-86d3-43c5-89a7-542506a74942'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:12.708469 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-df56a53f-86d3-43c5-89a7-542506a74942 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3653c18f26474021a165ad3573ea59fa', 'request_id': 'req-df56a53f-86d3-43c5-89a7-542506a74942'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:12.722920 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-df56a53f-86d3-43c5-89a7-542506a74942 admin admin] The token cache is being invalidate because project 3653c18f26474021a165ad3573ea59fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:56:12.723211 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-df56a53f-86d3-43c5-89a7-542506a74942 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:12.723840 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-df56a53f-86d3-43c5-89a7-542506a74942 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:12.725409 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1487/2943] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:56:12 2022] DELETE /identity/v3/projects/3653c18f26474021a165ad3573ea59fa => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:12.730329 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d3e6a-c40e-48f2-8eef-4fcec359db24 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:12.730765 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d3e6a-c40e-48f2-8eef-4fcec359db24 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:12.730903 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441d3e6a-c40e-48f2-8eef-4fcec359db24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:12.751067 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-441d3e6a-c40e-48f2-8eef-4fcec359db24 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:56:12.752327 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-441d3e6a-c40e-48f2-8eef-4fcec359db24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:12.815078 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1457/2944] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:56:12.824043 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a782a4-05cb-487e-b62b-0094aa20c985 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:12.824332 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a782a4-05cb-487e-b62b-0094aa20c985 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:12.824605 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a782a4-05cb-487e-b62b-0094aa20c985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:12.845425 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-85a782a4-05cb-487e-b62b-0094aa20c985 None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:56:12.846826 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-85a782a4-05cb-487e-b62b-0094aa20c985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:12.854665 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1488/2945] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:56:12.859083 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d3e6a-c40e-48f2-8eef-4fcec359db24 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:12.869034 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441d3e6a-c40e-48f2-8eef-4fcec359db24 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:12.874239 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d356a-93fb-40fc-a524-cdb33db4adb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:12.874889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d356a-93fb-40fc-a524-cdb33db4adb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:12.875176 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d356a-93fb-40fc-a524-cdb33db4adb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:12.875453 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d356a-93fb-40fc-a524-cdb33db4adb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:12.876219 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d356a-93fb-40fc-a524-cdb33db4adb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 09:56:12.876528 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d356a-93fb-40fc-a524-cdb33db4adb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:12.878166 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d356a-93fb-40fc-a524-cdb33db4adb8 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:12.892336 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1458/2946] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jul 21 09:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 09:56:12.896681 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a782a4-05cb-487e-b62b-0094aa20c985 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:12.905966 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a782a4-05cb-487e-b62b-0094aa20c985 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:12.910676 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034c54c-8c63-47cc-acb8-641977ca7163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:12.911313 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034c54c-8c63-47cc-acb8-641977ca7163 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:12.911592 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034c54c-8c63-47cc-acb8-641977ca7163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:12.911866 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034c54c-8c63-47cc-acb8-641977ca7163 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:12.912587 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034c54c-8c63-47cc-acb8-641977ca7163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:12.914473 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034c54c-8c63-47cc-acb8-641977ca7163 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:12.940014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1489/2947] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jul 21 09:56:12 2022] POST /identity/v3/projects => generated 361 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 09:56:12.944318 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d356a-93fb-40fc-a524-cdb33db4adb8 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:12.955475 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d356a-93fb-40fc-a524-cdb33db4adb8 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:13.012800 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1db423-e07b-4226-8e73-baa28a2fbaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:13.013467 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1db423-e07b-4226-8e73-baa28a2fbaf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:13.014367 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1db423-e07b-4226-8e73-baa28a2fbaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:13.014900 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1db423-e07b-4226-8e73-baa28a2fbaf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:13.015481 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1db423-e07b-4226-8e73-baa28a2fbaf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:13.017461 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1db423-e07b-4226-8e73-baa28a2fbaf8 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:13.037181 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1459/2948] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jul 21 09:56:12 2022] POST /identity/v3/users => generated 281 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 09:56:13.041745 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034c54c-8c63-47cc-acb8-641977ca7163 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:13.053513 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034c54c-8c63-47cc-acb8-641977ca7163 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:13.058204 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab19878-3c54-4f36-a71b-98f8decda863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:13.058893 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab19878-3c54-4f36-a71b-98f8decda863 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:13.059178 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab19878-3c54-4f36-a71b-98f8decda863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:13.059577 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab19878-3c54-4f36-a71b-98f8decda863 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:13.061003 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab19878-3c54-4f36-a71b-98f8decda863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:13.062781 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab19878-3c54-4f36-a71b-98f8decda863 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:13.073797 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1490/2949] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:56:13.078995 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1db423-e07b-4226-8e73-baa28a2fbaf8 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:13.090857 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1db423-e07b-4226-8e73-baa28a2fbaf8 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:13.096227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee714c-31f1-4b69-a69f-e4f2cc9b329f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:13.096838 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee714c-31f1-4b69-a69f-e4f2cc9b329f admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:13.097122 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee714c-31f1-4b69-a69f-e4f2cc9b329f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:13.097400 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee714c-31f1-4b69-a69f-e4f2cc9b329f admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:13.098136 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee714c-31f1-4b69-a69f-e4f2cc9b329f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:13.099766 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee714c-31f1-4b69-a69f-e4f2cc9b329f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:13.111352 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1460/2950] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jul 21 09:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 09:56:13.115486 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab19878-3c54-4f36-a71b-98f8decda863 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:13.126392 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab19878-3c54-4f36-a71b-98f8decda863 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:13.130981 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53735f24-49cc-4cbb-9cbb-3b6b85387433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:13.131848 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53735f24-49cc-4cbb-9cbb-3b6b85387433 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:13.132164 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53735f24-49cc-4cbb-9cbb-3b6b85387433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:13.132438 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53735f24-49cc-4cbb-9cbb-3b6b85387433 admin admin] PATH_INFO: `/v3/projects/a6aed74b900e4ae988082b9bb35f3732/users/512e024f99624453967d299db3c554c2/roles/3d7c80cf23864f58bf21f893ba7691d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:13.147019 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53735f24-49cc-4cbb-9cbb-3b6b85387433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6aed74b900e4ae988082b9bb35f3732, user_id=512e024f99624453967d299db3c554c2, role_id=3d7c80cf23864f58bf21f893ba7691d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:13.149741 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53735f24-49cc-4cbb-9cbb-3b6b85387433 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:13.164117 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1491/2951] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jul 21 09:56:13 2022] PUT /identity/v3/projects/a6aed74b900e4ae988082b9bb35f3732/users/512e024f99624453967d299db3c554c2/roles/3d7c80cf23864f58bf21f893ba7691d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:13.175376 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b010e7-6f11-4a62-a1d2-64f21554b1d8 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:13.175670 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b010e7-6f11-4a62-a1d2-64f21554b1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:13.175981 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b010e7-6f11-4a62-a1d2-64f21554b1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:13.204889 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b3b010e7-6f11-4a62-a1d2-64f21554b1d8 None None] MFA Rules not processed for user `512e024f99624453967d299db3c554c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:56:13.206227 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b3b010e7-6f11-4a62-a1d2-64f21554b1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:13.248791 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:56:13.276032 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1461/2952] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:56:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:56:13.308238 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53735f24-49cc-4cbb-9cbb-3b6b85387433 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:13.309938 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-53735f24-49cc-4cbb-9cbb-3b6b85387433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:13.367717 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53735f24-49cc-4cbb-9cbb-3b6b85387433 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:13.371263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521a8b3-4366-41a3-8b62-e16c454e34ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a3b3421e9d8427388f4fd2be5a601cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:13.371934 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521a8b3-4366-41a3-8b62-e16c454e34ef service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:13.372236 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521a8b3-4366-41a3-8b62-e16c454e34ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:13.372514 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521a8b3-4366-41a3-8b62-e16c454e34ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:13.381894 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521a8b3-4366-41a3-8b62-e16c454e34ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:13.383331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521a8b3-4366-41a3-8b62-e16c454e34ef service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:13.393264 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1492/2953] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:56:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:56:13.408745 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b010e7-6f11-4a62-a1d2-64f21554b1d8 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:13.473056 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b010e7-6f11-4a62-a1d2-64f21554b1d8 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:13.477625 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de88c4f-895f-4975-918e-ec03dcc62853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:13.478490 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de88c4f-895f-4975-918e-ec03dcc62853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:13.478775 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de88c4f-895f-4975-918e-ec03dcc62853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:13.479053 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de88c4f-895f-4975-918e-ec03dcc62853 admin admin] PATH_INFO: `/v3/users/512e024f99624453967d299db3c554c2` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:13.480279 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de88c4f-895f-4975-918e-ec03dcc62853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=512e024f99624453967d299db3c554c2)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:13.482230 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de88c4f-895f-4975-918e-ec03dcc62853 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:13.535956 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9de88c4f-895f-4975-918e-ec03dcc62853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '512e024f99624453967d299db3c554c2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:13.543293 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9de88c4f-895f-4975-918e-ec03dcc62853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '512e024f99624453967d299db3c554c2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:13.550424 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9de88c4f-895f-4975-918e-ec03dcc62853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '512e024f99624453967d299db3c554c2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:13.562906 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9de88c4f-895f-4975-918e-ec03dcc62853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '512e024f99624453967d299db3c554c2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:13.563553 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9de88c4f-895f-4975-918e-ec03dcc62853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '512e024f99624453967d299db3c554c2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:13.569470 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1462/2954] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jul 21 09:56:13 2022] DELETE /identity/v3/users/512e024f99624453967d299db3c554c2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:13.573997 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b057e-ff06-45d6-a100-5897862bffed None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:13.574475 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b057e-ff06-45d6-a100-5897862bffed None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:13.574573 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b057e-ff06-45d6-a100-5897862bffed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:13.592798 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-999b057e-ff06-45d6-a100-5897862bffed None None] MFA Rules not processed for user `af6fa97a676b4e53bc4e27a552d09f11`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 09:56:13.594014 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-999b057e-ff06-45d6-a100-5897862bffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:13.620148 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:56:13.657858 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1493/2955] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jul 21 09:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 09:56:13.667819 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de88c4f-895f-4975-918e-ec03dcc62853 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:13.669269 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9de88c4f-895f-4975-918e-ec03dcc62853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:13.728300 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de88c4f-895f-4975-918e-ec03dcc62853 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:13.731950 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85ccac-f890-4442-999b-a77778bf303a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e2a13c2cb864175a082de3ca8b166c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:13.732983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85ccac-f890-4442-999b-a77778bf303a service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:13.732983 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85ccac-f890-4442-999b-a77778bf303a service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:13.733168 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85ccac-f890-4442-999b-a77778bf303a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:13.742183 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85ccac-f890-4442-999b-a77778bf303a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:13.743646 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85ccac-f890-4442-999b-a77778bf303a service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:13.754576 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1463/2956] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:56:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:56:14.042130 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b057e-ff06-45d6-a100-5897862bffed None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:56:14.043704 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-999b057e-ff06-45d6-a100-5897862bffed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:56:14.056451 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b057e-ff06-45d6-a100-5897862bffed None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:56:14.062566 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441144b0-c68d-43fe-8c53-0fb21dccbb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af6fa97a676b4e53bc4e27a552d09f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a27dd5dc3697487bbb9b172396ff08c2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:56:14.063436 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441144b0-c68d-43fe-8c53-0fb21dccbb39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:56:14.063717 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441144b0-c68d-43fe-8c53-0fb21dccbb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:56:14.064042 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441144b0-c68d-43fe-8c53-0fb21dccbb39 admin admin] PATH_INFO: `/v3/projects/a6aed74b900e4ae988082b9bb35f3732` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:56:14.065263 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441144b0-c68d-43fe-8c53-0fb21dccbb39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6aed74b900e4ae988082b9bb35f3732)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:56:14.067331 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441144b0-c68d-43fe-8c53-0fb21dccbb39 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:56:14.114682 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-441144b0-c68d-43fe-8c53-0fb21dccbb39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6aed74b900e4ae988082b9bb35f3732', 'request_id': 'req-441144b0-c68d-43fe-8c53-0fb21dccbb39'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:14.120020 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-441144b0-c68d-43fe-8c53-0fb21dccbb39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6aed74b900e4ae988082b9bb35f3732', 'request_id': 'req-441144b0-c68d-43fe-8c53-0fb21dccbb39'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:14.126773 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-441144b0-c68d-43fe-8c53-0fb21dccbb39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6aed74b900e4ae988082b9bb35f3732', 'request_id': 'req-441144b0-c68d-43fe-8c53-0fb21dccbb39'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:14.139218 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-441144b0-c68d-43fe-8c53-0fb21dccbb39 admin admin] The token cache is being invalidate because project a6aed74b900e4ae988082b9bb35f3732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 09:56:14.139493 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-441144b0-c68d-43fe-8c53-0fb21dccbb39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:14.140142 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-441144b0-c68d-43fe-8c53-0fb21dccbb39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 09:56:14.141679 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1494/2957] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jul 21 09:56:14 2022] DELETE /identity/v3/projects/a6aed74b900e4ae988082b9bb35f3732 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 09:56:23.750063 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:56:24.136045 dv-cinder-826608-6-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 09:59:23.273412 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85ccac-f890-4442-999b-a77778bf303a service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 09:59:23.275384 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-af85ccac-f890-4442-999b-a77778bf303a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:59:23.382306 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85ccac-f890-4442-999b-a77778bf303a service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 09:59:23.444946 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184211c-8967-44c6-8bc4-c62947ed2cd2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72f9ef6de66e456e800237e9f954e98d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fe0f489cbce43859f050c3270e00709', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 09:59:23.445595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184211c-8967-44c6-8bc4-c62947ed2cd2 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 09:59:23.445884 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184211c-8967-44c6-8bc4-c62947ed2cd2 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 09:59:23.446320 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184211c-8967-44c6-8bc4-c62947ed2cd2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 09:59:23.448165 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f184211c-8967-44c6-8bc4-c62947ed2cd2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 09:59:23.459626 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184211c-8967-44c6-8bc4-c62947ed2cd2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 09:59:23.461282 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184211c-8967-44c6-8bc4-c62947ed2cd2 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 09:59:23.471595 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1464/2958] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jul 21 09:59:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 09:59:33.467258 dv-cinder-826608-6-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=71 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}